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A text-to-speech system includes a memory 
storing a set of quantization vectors. A first pro- 
cessing module is responsive to the sound segment 
codes generated in response to text in the sequence 
to identify strings of noise compensated quantization 
vectors for respective sound segment codes in the se- 
quence. A decoder generates a speech data sequence 
in response to the strings of quantization vectors. An 
audio transducer is coupled to the processing mod- 
ules, and generates sound in response to the speech 
data sequence. The quantization vectors represent a 
quantization of a sound segment data having a pre- 
emphasis to de-correlate the sound samples used for 
quantization and the quantization noise. In decom- 
pressing the sound segment data, an inverse linear 
prediction filter is applied to the identified strings 
of quantization vectors to reverse the pre-emphasis. 
Also, the quantization vectors represent quantization 
of results of pitch filtering of sound segment data. 
Thus, an inverse pitch filter is applied to the identi- 
fied strings of quantization vectors in the module for 
generating the speech data sequence. 



r 



CPU 



13 



DISPLAY 



AUD1D 
DUT 



^-17 



DISK 



12 



KEYBOARD 



ENCODED 




VOICE 




TABLES 




TEXT- 




TO- 




SPEECH 




CODE 




BUFFERS 


OTHER 




HOST 




MEMORY 





• TTS 

DICTIONARY 



• DIPHONE 
TABLE 



• NOISE SHAPED 
VECTOR QUANT- 
IZATION TABLE 

m FOR ENCODING 

•NOISE COMPENSA- 
TED VECTOR 
TABLE FOR 
DECODING 



•IS 



FOR THE PURPOSES OF INFORMATION ONLY 

Codes used to identify States party to the PCT on the front pages of pamphlets publishing international 
applications under the PCT. 



AT 


Austria 


GB 


United Kingdom 


MR 


Mauritania 


At 


Australia 


GE 


Georgia 


MW 


Malawi 


BB 


Barbados 


GN 


Guinea 


NE 


Niger 


BE 


Belgium 


GR 


Greece 


NL 


Netherlands 


BF 


Burkina Fa bo 


HTJ 


Hungary 


NO 


Norway 


BG 


Bulgaria 


IE 


Ireland 


■ NZ 


New Zealand 


BJ 


Benin 


IT 


Italy 


PL 


Poland 


BR 


Brazil 


JP 


Japan 


PT 


Portugal 
Romania 


BY 


Belarus 


KE 


Kenya 


RO 


CA 


Canada 


KG 


Kyrgystan 


RU 


Russian Federation 


CF 


Central African Republic 


KP 


Democratic People' i Republic 


SD 


Sudan 


CG 


Congo 




of Korea 


SB 


Sweden 


CH 


Switzerland 


KR 


Republic of Korea 


SI 


Slovenia 


ci 


Cote d' I voire 


KZ 


Kazakhstan 


SK 


Slovakia ' 


CM 


Cameroon 


U 




SN 


Senegal 
Chad 


CN 


China 


LK 


Sri Lanka 


TO 


CS 




LU 




TG 


Togo 


CZ 


Czech Republic 


LV 


Latvia 


TJ 


Tajikistan 


DC 


Germany 


MC 


Monaco 


TT 


Trinidad and Tobago 
Ukraine 


DK 


Denmark 


MD 


Republic of Moldova 


UA 


ES 


Spain 


MG 


Madagascar 


US 


United States of America 


n 


Finland 


ML 


Man 


UZ 


Uzbekistan 


FR 


France 


MN 


Mongolia 


VN 


Viet Nam 


GA 


Gabon 











WO 94/17518 



PCT/US94/00649 



- 1 - 

TEXT-TO-SPEECH SYSTEM USING VECTOR QUANTIZATION 
BASED SPEECH ENCODING/DECODING 



LIMITED COPYRIGHT WAIVER 
A portion of the disclosure of this patent document contains 
5 material to which the claim of copyright protection is made. The 
copyright owner has no objection to the facsimile reproduction by any 
person of the patent document or the patent disclosure, as it appears in 
the U.S. Patent and Trademark Office file or records, but reserves all 
other rights whatsoever. 



10 BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to translating text in a computer 
system to synthesized speech; and more particularly to techniques used 
in such systems for storage and retrieval of speech data. 



15 Description pf the Relate^ Art 

In text-to-speech systems, stored text in a computer is translated 
to synthesized speech. As can be appreciated, this kind of system 
would have wide spread application if it were of reasonable cost. For 
instance, a text-to-speech system could be used for reviewing electronic 

20 mail remotely across a telephone line, by causing the computer storing 
the electronic mail to synthesize speech representing the electronic mail. 
Also, such systems could be used for reading to people who are visually 
impaired. In the word processing context, text-to-speech systems might 
be used to assist in proofreading a large document. 

25 However in prior art systems which have reasonable cost, the 

quality of the speech has been relatively poor making it uncomfortable 
to use or difficult to understand. In order to achieve good quality 
spe ch, prior art speech synthesis systems need specialized hardware 
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which is very expensive, and/or a large amount of memory space in the 
computer system generating the sound. 

In text-to-speech systems, an algorithm reviews an input text 
string, and translates the words in the text string into a sequence of 
5 diphones which must be translated into synthesized speech. Also, text- 
to-speech systems analyze the text based on word type and context to 
generate intonation control used for adjusting the duration of the sounds 
and the pitch of the sounds involved in the speech. 

Diphones consist of a unit of speech composed of the transition 

10 between one sound, or phoneme, and an adjacent sound, or phoneme. 
Diphones typically start at the center of one phoneme and end at the 
center of a neighboring phoneme. This preserves the transition between 
the sounds relatively well. 

American English based text-to-speech systems, depending on the 

1 5 particular implementation, use about fifty different sounds referred to as 
phones. Of these fifty different sounds, the standard language uses 
about 1 800 diphones out of possible 2500 phone pairs. Thus, a text-to- 
speech system must be capable of reproducing 1800 diphones. To 
store the speech data directly for each diphone would involve a huge 

20 amount of memory. Thus, compression techniques have evolved to limit 
the amount of memory required for storing the diphones. However, to 
be successful, the computational complexity of the decoder for 
decompressing the diphone data must be very low so that the system 
is capable of running across a broad range of hardware platforms with 

25 very high quality reproduction. 

Prior art systems which have addressed this problem are 
described in part in United States Patent No. 8,452,168, entitled 
COMPRESSION OF STORED WAVE FORMS FOR ARTIFICIAL SPEECH, 
invented by Sprague; and United States Patent No. 4,692,941 , entitled 

30 REAL-TIME TEXT-TO-SPEECH CONVERSION SYSTEM, invented by 
Jacks, et al. Further background concerning speech synthesis may be 
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found in United States Patent No. 4,384,169, entitled METHOD AND 
APPARATUS FOR SPEECH SYNTHESIZING, invented by Mozer, et al. 

Notwithstanding the prior work in this area, the use of text-to- 
speech systems has not gained widespread acceptance. It is desireable 
5 therefore to provide a software only text-to-speech system which is 
portable to a wide variety of microcomputer platforms, and conserves 
memory space in such platforms for other uses. 



SUMMARY OF THE INVENTION 
The present invention provides a software only real time, text-to- 
10 speech system suitable for application a wide variety of personal 
computer platforms which uses a relatively small amount of host system 
memory for execution. The system is based on a speech compression 
algorithm which takes advantage of certain specialized knowledge 
concerning speech including the following: 
15 1 ) Adjacent samples of the speech data are highly correlated. 

Thus a fixed linear prediction filter may be used to partially 
remove the correlation between adjacent samples. 

2) In the case of voice to speech (e.g., vowels, nasals, etc.), 
the speech wave forms can be regarded as slowly varying 

20 periodic signals. Thus, an adaptive pitch predictor can be 

used to remove the redundancy in speech data and achieve 
a high data compression. 

3) Finally, vector quantization is an extremely efficient 
approach to code correlated data vectors. It can be applied 

25 to partially de-correlated speech data according to the 

present invention, and noise shaping can be incorporated 
into the vector quantization process to improve the 
subjective quality of the synthesized speech. Further, a 
variety of different compression rates can be achieved by 

30 simply varying the vector size used for vector quantization . 
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Thus, according to one aspect, the invention can be characterized 
as an apparatus for synthesizing speech in response to a sequence of 
sound segment codes representing speech. The system includes a 
memory storing a set of noise compensated quantization vectors. A 
5 processing module in the apparatus is responsive to the sound segment 
codes in the sequence to identify strings of noise compensated 
quantization vectors in the set for respective sound segment codes in 
the sequence. A second processing module generates a speech data 
sequence in response to the strings of noise compensated quantization 

10 vectors. Finally, an audio transducer is coupled to the processing 
modules, and generates sound in response to the speech data sequence. 

For noise compensation according to this aspect, sounds are 
encoded using noise shaped data and first set of quantization vectors 
adapted for the noise shaped data. In decoding, a second set of noise 

1 5 compensated vectors different from the first set are used to recover 
improved quality sound. 

Another aspect of the invention involves utilizing the quantization 
vectors to represent filtered sound segment data, and providing for a 
module for applying an inverse filter to the strings of quantization 

20 vectors in the generation of the speech data sequence. According to 
this aspect, the quantization vectors may represent a quantization of 
results of linear prediction filtering of sound segment data for spectral 
flattening to de-correlate the sound samples used for quantization and 
the quantization noise. In decompressing the sound segment data, an 

25 inverse linear prediction filter is applied to the identified strings of 
quantization vectors to recover the sound data. Also, the quantization 
vectors represent quantization of results of pitch filtering of sound 
segment data. Thus, an inverse pitch filter is applied to the identified 
strings of quantization vectors in the module of generating the speech 

30 data sequence. 
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In systems using the inverse linear prediction filter and the inverse 
pitch filter, the sound segment codes also include parameters used in 
executing the inverse filtering steps. In the preferred system, these 
parameters are chosen, along with filter coefficients used in the 
5 decoding, so that the decoding can be executed without multiplication. 
That is, shifts and adds replace any multiplication required by these 
specifically chosen values. 

The invention can also be characterized as an apparatus for 
synthesizing speech in response to text. This system includes a module 

1 0 that translates received text into a sequence of sound segments codes 
which are decoded as described above. The text translator includes a 
table of encoded diphones having entries that include data identifying a 
string of quantization vectors in the set for the respective diphones. The 
sequence of sound segment codes thus comprises a sequence of indices 

15 to the table of encoded diphones representing the text. The strings of 
the quantization vectors for a given sound segment code are identified 
by accessing the entries in the table of encoded diphones. 

The module for generating the speech data waveform may also 
include modules for improving the quality of the synthesized speech. 

20 Such modules include a routine for blending the ending of a particular 
diphone in the sequence with beginning of an adjacent diphone to 
smooth discontinuities between the particular and adjacent diphone data 
strings. Further, the string of quantized speech data may be applied to 
a system which adjusts the pitch and duration of the sounds represented 

25 by the strings of quantization vectors. 

According to yet another aspect of the invention, the apparatus 
for synthesizing speech may include an encoder for generating the table 
of encoded diphones. In this aspect, the encoder receives sampled 
speech for the respective diphones, applies a fixed linear prediction filter 

30 to partially de-correlate the speech samples and the quantization noise, 
applies a pitch filter to the output of the linear prediction filter, and 
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applies a noise shaping filter to generate a resulting set of vectors. The 
resulting set of vectors is then matched to vectors in a vector 
quantization table. The vectors in the vector quantization table are 
related to the quantization vectors used for decoding the speech data by 
5 the same noise shaping filter or a derivative of it to subjectively improve 
the quality of the decompressed speech. 

This encoding technique allows use of the decoding technique 
which is very simple, requires a small amount of memory, and produces 
very high quality speech. 

10 Accordingly, the present invention is concerned with a speech 

compression/decompression technique for use in a text-to-speech 
system in which a higher level of compression is achieved while keeping 
the decoder complexity to an absolute minimum. The compression ratio 
can be varied depending on the available RAM in the computer. In order 

15 to store speech in an uncompressed form, normally 8-16 bits per sample 
is required. Using the speech compression technique of the present 
invention, the number of bits required to store each sample can be 
reduced to 0.5 bits (i.e., about 16 samples of speech can be stored 
using 8 bits of memory). However, higher quality synthesized speech 

20 can be produced when larger RAM space is available, using about 4 bits 
per sample. 

Other aspects and advantages of the present invention can be 
seen upon review of the figures, the detailed description and the claims 
which follow. 

25 BRIEF DESCRIPTION OF THE FIGURES 

Fig. 1 is a block diagram of a generic hardware platform 

incorporating the text-to-speech system of the present invention. 

Fig. 2 is a flow chart illustrating the basic text-to-speech routine 

according to the present invention. 
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Fig. 3 illustrates the format of diphone records according to one 
embodiment of the present invention i 

Fig. 4 is a flow chart illustrating the encoder for speech data 
according to the present invention. 
5 Fig. 5 is a graph discussed in reference to the estimation of pitch 

filter parameters in the encoder of Fig. 4. 

Fig. 6 is a flow chart illustrating the full search used in the 
encoder of Fig. 4. 

Fig. 7 is a flow chart illustrating a decoder for speech data 
10 according to the present invention. 

Fig. 8 is a flow chart illustrating a technique for blending the 
beginning and ending of adjacent diphone records. 

Fig. 9 consists of a set of graphs referred to in explanation of the 
blending technique of Fig. 8. 
15 Fig. 10 is a graph illustrating a typical pitch versus time diagram 

for a sequence of frames of speech data. 

Fig. 11 is a flow chart illustrating a technique for increasing the 
pitch period of a particular frame. 

Fig. 12 is a set of graphs referred to in explanation of the 
20 technique of Fig. 1 1 . 

Fig. 13 is a flow chart illustrating a technique for decreasing the 
pitch period of a particular frame. 

Fig. 14 is a set of graphs referred to in explanation of the 
technique of Fig. 13. 
25 Fig. 1 5 is a flow chart illustrating a technique for inserting a pitch 

period between two frames in a sequence. 

Fig. 16 is a set of graphs referred to in explanation of the 
technique of Fig. 15. 

Fig. 17 is a flow chart illustrating a technique for deleting a pitch 
30 period in a sequence f frames. 
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Fig. 18 is a set of graphs referred to in explanation of the 
technique of Fig. 17. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
A detailed description of preferred embodiments of the present 
invention is provided with reference to the figures. Figs. 1 and 2 
provide a overview of a system incorporating the present invention. Fig. 
3 illustrates the basic manner in which diphone records are stored 
according to the present invention. Figs. 4-6 illustrate the encoding 
methods based on vector quantization of the present invention. Fig. 7 
illustrates the decoding algorithm according to the present invention. 

Figs. 8 and 9 illustrate a preferred technique for blending the 
beginning and ending of adjacent diphone records. Figs. 10-18 illustrate 
the techniques for controlling the pitch and duration of sounds in the 
text-to-speech system. 

I. System Overview (Figs. 1-3) 

Fig. 1 illustrates a basic microcomputer platform incorporating a 
text-to-speech system based on vector quantization according to the 
present invention. The platform includes a central processing unit 10 
coupled to a host system bus 11. A keyboard 1 2 or other text input 
device is provided in the system. Also, a display system 13 is coupled 
to the host system bus. The host system also includes a non-volatile 
storage system such as a disk drive 14. Further, the system includes 
host memory 1 5. The host memory includes text-to-speech (TTS) code, 
including encoded voice tables, buffers, and other host memory. The 
text-to-speech code is used to generate speech data for supply to an 
audio output module 16 which includes a speaker 17. 

According to the present invention, the encoded voice tables 
include a TTS dictionary which is used to translate text to a string of 
diphones. Also included is a diphone table which translates the 
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diphones to identified strings of quantization vectors. A quantization 
vector table is used for decoding the sound segment codes of the 
diphone table into the speech data for audio output. Also, the system 
may include a vector quantization table for encoding which is loaded into 
5 the host memory 1 5 when necessary. 

The platform illustrated in Fig. 1 represents any generic 
microcomputer system, including a Macintosh based system, an DOS 
based system, a UNIX based system or other types of microcomputers. 
The text-to-speech code and encoded voice tables according to the 

1 0 present invention for decoding occupy a relatively small amount of host 
memory 1 5. For instance, a text-to-speech decoding system according 
to the present invention may be implemented which occupies less than 
640 kilobytes of main memory, and yet produces high quality, natural 
sounding synthesized speech. 

15 The basic algorithm executed by the text-to-speech code is 

illustrated in Fig. 2. The system first receives the input text (block 20). 
The input text is translated to diphone strings using the TTS dictionary 
(block 21). At the same time, the input text is analyzed to generate 
intonation control data, to control the pitch and duration of the diphones 

20 making up the speech (block 22). 

After the text has been translated to diphone strings, the diphone 
strings are decompressed to generate vector quantized data frames 
(block 23). After the vector quantized (VQ) data frames are produced, 
the beginnings and endings of adjacent diphones are blended to smooth 

25 any discontinuities (block 24). Next, the duration and pitch of the 
diphone VQ data frames are adjusted in response to the intonation 
control data (block 25 and 26). Finally, the speech data is supplied to 
the audio output system for real time speech production (block 27). For 
systems having sufficient processing power, an adaptive post filter may 

30 be applied to further improve the speech quality. 
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The TTS dictionary can be implemented using any one of a variety 
of techniques known in the art. According to the present invention, 
diphone records are implemented as shown in Fig. 3 in a highly 
compressed format. 
5 As shown in Fig. 3, records for a left diphone 30 and a record for 

a right diphone 31 are shown. The record for the left diphone 30 
includes a count 32 of the number NL of pitch periods in the diphone. 
Next, a pointer 33 is included which points to a table of length NL 
storing the number LP. for each pitch period, i goes from 0 to NL-1 of 
10 pitch values for corresponding compressed frame records. Finally, 
pointer 34 is included to a table 36 of ML vector quantized compressed 
speech records, each having a fixed set length of encoded frame size 
related to nominal pitch of the encoded speech for the left diphone. The 
nominal pitch is based upon the average number of samples for a given 
1 5 pitch period for the speech data base. 

A similar structure can be seen for the right diphone 31 . Using 
vector quantization, a length of the compressed speech records is very 
short relative to the quality of the speech generated. 

The format of the vector quantized speech records can be 
20 understood further with reference to the frame encoder routine and the 
frame decoder routine described below with reference to Figs. 4-7. 



II. The Encoder/Decoder Routines (Figs. 4-7) 

The encoder routine is illustrated in Fig. 4. The encoder accepts 
as input a frame of speech data. In the preferred system, the speech 

25 samples are represented as 12 or 16 bit two's complement numbers, 
sampled at 22,252 Hz. This data is divided into non-overlapping frames 
s n having a length of N, where N is referred to as the frame size. The 
value of N depends on the nominal pitch of the speech data. If the 
nominal pitch of the recorded speech is less than 165 samples (or 135 

30 Hz), the value of N is chosen to be 96. Otherwise a frame size of 1 60 
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is used. The encoder transforms the N-point data sequence s n into a 

byte stream of shorter length, which depends on the desired 

compression rate. For example, if N = 1 60 and very high data 

compression is desired, the output byte stream can be as short as 12 

5 eight bit bytes. A block diagram of the encoder is shown in Fig. 4. 

Thus, the routine begins by accepting a frame s n (block 50). To 

remove low frequency noise, such as DC or 60 Hz power line noise, and 

produce offset free speech data, signal s n is passed through a high pass 

filter. A difference equation used in a preferred system to accomplish 

10 this is set out in Equation 1 for 0^n<N. 

x = s - s . + 0.999 *x n . 
n n n-1 n-1 

Equation 1 

The value x n is the "offset free" signal. The variables s ^ and x 1 
are initialized to zero for each diphone and are subsequently updated 
1 5 using the relation of Equation 2. 

x -1 = X N and s -1 = S N 

Equation 2 

This step can be referred to as offset compensation or DC 
removal (block 51). 

20 In order to partially decorrelate the speech samples and the 

quantization noise, the sequence x n is passed through a fixed first order 
linear prediction filter. The difference equation to accomplish this is set 
forth in Equation 3. 

y n = x n - 0.875 * V1 

25 Equation 3 

The linear prediction filtering of Equation 3 produces a frame y 
(block 52). The filter parameter, which is equal to 0.875 in Equation 3, 
will have to be modified if a different speech sampling rate is used. The 
value of x ^ is initialized to zero for each diphone, but will be updated 

30 in the step of inverse linear prediction filtering (block 60) as described 
below. 
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It is possible to use a variety of filter types, including, for 

instance, an adaptive filter in which the filter parameters are dependent 

on the diphones to be encoded, or higher order filters. 

The sequence y n produced by Equation 3 is then utilized to 

5 determine an optimum pitch value, P^, and an associated gain factor, 

0. P is computed using the functions s (P), s (P), s (P), and the 
opx xy xx yy 

coherence function Coh(P) defined by Equations 4, 5, 6 and 7 as set out 
below. 

N-1 

10 V P, = Z V n * ""V - P + n 

n=0 max 

Equation 4 

N-1 



I 

15 n=0 



s (P) = 2. y * y 



Equation 5 



N-1 



s <P) - I PBUF p p n * PBUF p . p + n 
N = 0 max max 
20 Equation 6 

and 



Coh(P) = s (P) * s (P) / (s (P) * s (P)) 
xy xy xx yy 



Equation 7 



PBUF is a pitch buffer of size P , which is initialized to zero, 

max 

25 and updated in the pitch buffer update block 59 as described below. 

P opt is the value of p for which Coh(P) is maximum and s xy (P) is 
positive. The range of P considered depends on the nominal pitch of the 
speech being coded. The range is (96 to 350) if the frame size is equal 

to 96 and is (160 to 414) if the frame size is equal to 160. P is 

max 

30 350 if nominal pitch is less than 160 and is equal to 414 otherwise. 
The parameter P Qpt can be represented using 8 bits. 



9 
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The computation of P ^ can be understood with reference to Fig. 

5. In Fig. 5, the buffer PBUF is represented by the sequence 100 and 

the frame y n is represented by the sequence 101. In a segment of 

speech data in which the preceding frames are substantially equal to the 

5 frame y , PBUF and y will look as shown in Fig. 5. P _ will have the 

'n n opt 

value at point 102, where the vector y n 101 matches as closely as 
possible a corresponding segment of similar length in PBUF 100. 

The pitch filter gain parameter 0 is determined using the 
expression of Equation 8. 

Equation 8 

fi is quantized to four bits, so that the quantized value of 0 can 
range from 1/16 to 1, in steps of 1/16. 

Next, a pitch filter is applied (block 54). The long term 
1 5 correlations in the pre-emphasized speech data y n are removed using the 
relation of Equation 9. 

'n = V n -*«PBUF p +n . 0 £ n < N. 

max opt _ . 

Equation 9 

This results in computation of a residual signal r n - 
20 Next, a scaling parameter G is generated using a block gain 

estimation routine (block 55). In order to increase the computational 
accuracy of the following stages of processing, the residual signal r n is 
reseated. The scaling parameter, G, is obtained by first determining the 
largest magnitude of the signal r n and quantizing it using a 7-level 
25 quantizer. The parameter G can take one of the following 7 values: 
256, 512, 1024, 2048, 4096, 8192, and 16384. The consequence of 
choosing these quantization levels is that the rescaling operation can be 
implemented using only shift operations. 

Next the routine proceeds to residual coding using a full search 
30 vector quantization code (block 56). In order to code the residual signal 
r , the n point sequence r n is divided into non-overlapping blocks of 
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length M # wher M is referred to as the "vector size". Thus, M sample 
blocks b.. are created, wher i is an index from zero to M-1 on the block 
number, and j is an index from zero to N/M-1 on the sample within the 
block. Each block may be defined as set out in Equation 10. 
5 b^ = r Mj + . , (0 ^ i < N/M and j < 0 < M) 

Equation 10 

Each of these M sample blocks b~ will be coded into an 8 bit 
number using vector quantization. The value of M depends on the 
desired compression ratio. For example, with M equal to 1 6, very high 

10 compression is achieved (i.e., 1 6 residual samples are coded using only 
8 bits). However, the decoded speech quality can be perceived to be 
somewhat noisy with M = 16. On the other hand, with M = 2, the 
decompressed speech quality will be very close to that of uncompressed 
speech. However the length of the compressed speech records will be 

1 5 longer. The preferred implementation, the value M can take values 2, 
4, 8, and 16. 

The vector quantization is performed as shown in Fig. 6. Thus, 
for all blocks b.. a sequence of quantization vectors is identified (block 
120). First, the components of block b.. are passed through a noise 
20 shaping filter and scaled as set out in Equation 1 1 (block 121). 

w = 0.875 # w. 1 - 0.5 * w. 0 + 0.4375 * w. Q + b.., 

0 <; j < M 

v.. = G * w. 0 ^ j < M 

Equation 1 1 

25 Thus, v.. is the jth component of the vector v., and the values 

W -V w _2 and w -3 are the states of the noise shaping filter and are 
initialized to zero for each diphone. The filter coefficients are chosen to 
shape the quantization noise spectra in order to improve the subjective 
quality of the decompressed speech. After each vector is coded and 
30 decoded, these states are updated as described below with reference to 
blocks 124-126. 
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Next, the routine finds a p inter to the best match in a vector 
quantization table (block 122). The vector quantization table 123 
consists of a sequence of vectors C Q through C 2g5 (block 123). 

Thus, the vector v. is compared against 256 M-point vectors, 
5 which are precomputed and stored in the code table 123. The vector 
C . which is closest to v. is determined according to Equation 1 2. The 
value C p for p = 0 through 255 represents the p encoding vector from 
the vector quantization code table 123. 
M-1 

10 min Z (v.. - C .) 2 

. A U PJ 
P J = 0 



Equation 12 

The closest vector C q . can also be determined efficiently using the 
technique of Equation 13. 



15 v. T * C qj «s v. T # C p forallp(0<sp<s255) 

Equation 13 

In Equation 13, the value v T represents the transpose of the vector v, 

and "•" represents the inner product operation in the inequality. 

The encoding vectors C p in table 1 23 are utilized to match on the 

20 noise filtered value v... However in decoding, a decoding vector table 

125 is used which consists of a sequence of vectors QV . The values 

P 

QV p are selected for the purpose of achieving quality sound data using 

the vector quantization technique. Thus, after finding the vector C ., 

the pointer q is utilized to access the vector QV .. The decoded 

q» 

25 samples corresponding to the vector b. which is produced at step 55 of 

Fig. 4, is the M-point vector (1/G) * QV .. The vector C is related to 

qi p 

the vector QV p by the noise shaping filter operation of Equation 1 1 . 
Thus, when the decoding vector QV p is accessed, no inverse noise 
shaping filter needs to be computed in the decode operation. The table 
30 125 of Fig: 6 thus includes noise compensated quantization vectors. 
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In continuing to compute the encoding vectors for the vectors b.. 
which make up the residual signal r f the decoding vector of the pointer 
to the vector b. is accessed (block 124). That decoding vector is used 
for filter and PBUF updates (block 126). 
5 For the noise shaping filter, after the decoded samples are 

computed for each sub-block b., the error vector (b.-QV^.) is passed 
through the noise shaping filter as shown in Equation 14. 

W. = 0.875 * W. 1 - 0.5 # W. 0 + 0^4375 * W. Q + [b.. - 

QV qj (j>] 
10 0 <; j < M 

Equation 14 

In Equation 1 4, the value QV qj (j) represents the j component of 
the decoding vector QV^.. The noise shaping filter states for the next 
block are updated as shown in Equation 15. 

15 W -1 - W M-1 

W -2 = W M-2 
w -3 = w M-3 

Equation 15 

This coding and decoding is performed for all of the N/M sub- 
20 blocks to obtain N/M indices to the decoding vector table 125. This 
string of indices Q n# for n going from zero to N/M-1 represent identifiers 
for a string of decoding vectors for the residual signal r . 

Thus, four parameters represent the N-point data sequence y n : 
1) Optimum pitch, P Qpt (8 bits), 
25 2) Pitch filter gain, 0 (4 bits), 

3) Scaling parameter, G (3 bits), and 

4) A string of decoding table indices, Q R (0 n < N/M). 
The parameters p and G can be coded into a single byte. Thus, 

only (N/M) plus 2 bytes are used to represent N samples of speech. For 
30 example, suppose nominal pitch is 100 samples long, and M = 16. In 
this case, a frame of 96 samples of speech are represented by 8 bytes: 
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1 byte for P , 1 byte f r 0 and G, and 6 bytes for the decoding table 
indices Q.^. If the uncompressed speech consists of 1 6 bit samples, 
then this represents a compression of 24:1. 

Back to Fig. 4, four parameters identifying the speech data are 
5 stored (block 57). In a preferred system, they are stored in a structure 
as described with respect to Fig. 3 where the structure of the frame can 
be characterized as follows: 

#define NumOfVectorsPerFrame (FrameSize / VectorSize) 

struct frame { 
10 unsigned Gain : 4; 

unsigned Beta : 3; 
unsigned UnusedBit: 1 ; 
unsigned char Pitch ; 

unsigned char VQcodes[NumOfVectorsPerFrame]; }; 

1 5 The diphone record of Fig. 3 utilizing this frame structure can be 

characterized as follows: 

DiphoneRecord 
{ 

char LeftPhone, RightPhone; 
20 short LeftPitchPeriodCount,RightPitchPeriodCount; 

short *LeftPeriods, *RightPeriods; 
struct frame *LeftData, *RightData; 

} 

These stored parameters uniquely provide for identification of the 
25 diphones required for text-to-speech synthesis. 

As mentioned above with respect to Fig. 6, the encoder continues 
decoding the data being encoded in order to update the filter and PBUF 
values. The first step involved in this is an inverse pitch filter (block 
58). With the vector r' corresponding to the decoded signal formed by 
30 concatenating the string of decoding vectors to represent the residual 
signal r f , the inverse filter is implemented as set out in Equation 16. 

V'n - r 'n + * # PBUF Pmax - Popt + n 0 * n < N " 

Equation 16 
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Next, the pitch buffer is updated (block 59) with the output of the 
inverse pitch filter. The pitch buffer PBUF is updated as set out in 
Equation 17. 

PBUF n = PBUF (n + N) 0^n<(P max -N) 

5 PBUF (Pmax - N + n) = V' n Osn<N 

Equation 17 

Finally, the linear prediction filter parameters are updated using an 
inverse linear prediction filter step (block 60). The output of the inverse 
pitch filter is passed through a first order inverse linear prediction filter 
10 to obtain the decoded speech. The difference equation to implement 
this filter is set out in Equation 18. 



x' = 0.875 * x' + y' 
n n-i n 



Equation 18 



In Equation 18 f x' is the decompressed speech. From this, the 
1 5 value of x ^ for the next frame is set to the value x^ for use in the step 
of block 52. 

Fig. 7 illustrates the decoder routine. The decoder module 
accepts as input (N/M) + 2 bytes of data, generated by the encoder 
module, and applies as output N samples of speech. The value of N 
20 depends on the nominal pitch of the speech data and the value of M 
depends on the desired compression ratio. 

In software only text-to-speech systems, the computational 
complexity of the decoder must be as small as possible to ensure that 
the text-to-speech system can run in real time even on slow computers. 
25 A block diagram of the encoder is shown in Fig. 7. 

The routine starts by accepting diphone records at block 200. 
The first step involves parsing the parameters G, /?, P , and the vector 
quantization string Q n (block 201). Next, the residual signal r' is 
decoded (block 202). This involves accessing and concatenating the 
30 decoding vectors for the vector quantization string as shown 
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schematically at block 203 with access to the decoding quantization 
vector table 125. 

After the residual signal r' is decoded, an inverse pitch filter is 
applied (block 204). This inverse pitch filter is implemented as shown 
5 in Equation 19: 

V' = r 'n + 0* SPBUF < P m*v " P nnt + 0<n<N. 

n n max opt 

Equation 19 

SPBUF is a synthesizer pitch buffer of length P initialized as zero for 

max 

each diphone, as described above with respect to the encoder pitch 
10 buffer PBUF. 

For each frame, the synthesis pitch buffer is updated (block 205). 
The manner in which it is updated is shown in Equation 20: 

SPBUF n = SPBUF (n + N) 0*n«P max -N) 

SPBUF (Pmax - N + n) = y'n 0sn<N 
1 5 Equation 20 

After updating SPBUF, the sequence y' is applied to an inverse 

linear prediction filtering step (block 206). Thus, the output of the 

inverse pitch filter y # is passed through a first order inverse linear 

prediction filter to obtain the decoded speech. The difference equation 

20 to implement the inverse linear prediction filter is set out in Equation 21 : 



x' = 0.875 * x' + y' 
n n-l n 



Equation 21 



In Equation 21, the vector x' corresponds to the decompressed 
speech. This filtering operation can be implemented using simple shift 
25 operations without requiring any multiplication. Therefore, it executes 
very quickly and utilizes a very small amount of the host computer 
resources. 

Encoding and decoding speech according to the algorithms 
described above, provide several advantages over prior art systems. 
30 First, this technique offers higher speech compression rates with 
decoders simple enough to be used in the implementation of software 
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only text-to-speech systems n computer systems with low processing 
power. Second, the technique offers a very flexible trade-off between 
the compression ratio and synthesizer speech quality. A high-end 
computer system can opt for higher quality synthesized speech at the 
5 expense of a bigger RAM memory requirement. 

III. Waveform Blending For Discontinuity Smoothing (Figs. 8 and 9) 
As mentioned above with respect to Fig. 2, the synthesized 
frames of speech data generated using the vector quantization technique 
may result in slight discontinuities between diphones in a text string. 
10 Thus, the text-to-speech system provides a module for blending the 
diphone data frames to smooth such discontinuities. The blending 
technique of the preferred embodiment is shown with respect to Figs. 
8 and 9. 

Two concatenated diphones will have an ending frame and a 

1 5 beginning frame. The ending frame of the left diphone must be blended 

with the beginning frame of the right diphone without audible 

discontinuities or clicks being generated. Since the right boundary of 

the first diphone and the left boundary of the second diphone 

correspond to the same phoneme in most situations, they are expected 

20 to be similar looking at the point of concatenation. However, because 

the two diphone codings are extracted from different context, they will 

not look identical. This blending technique is applied to eliminate 

discontinuities at the point of concatenation. In Fig. 9, the last frame, 

referring here to one pitch period, of the left diphone is designated L n 

25 (0<n < PL) at the top of the page. The first frame (pitch period) of the 

right diphone is designated R (Osn<PR). The blending of L and R 

n n n 

according to the present invention will alter these two pitch periods only 
and is performed as discussed with reference to Fig. 8. The waveforms 
in Fig. 9 are chosen to illustrate the algorithm, and may not be 
30 representative of real speech data. 
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Thus, the algorithm as shown in Fig. 8 begins with receiving the 
left and right diphone in a sequence (block 300). Next, the last frame 
of the left diphone is stored in the buffer l_ n (block 301 ). Also, the first 
frame of the right diphone is stored in buffer F* n (block 302). 
5 Next, the algorithm replicates and concatenates the left frame L n 

to form extend frame (block 303). In the next step, the discontinuities 
in the extended frame between the replicated left frames are smoothed 
(block 304). This smoothed and extended left frame is referred to as El 

n 

in Fig. 9. 

1 0 The extended sequence El n (0 ^ n < PL) is obtained in the first step 

as shown in Equation 22: 

El p =L n n = 0,1,...,PL-1 

E, PL + n = L n n = 0 < 1 PL - 1 



15 Then discontinuity smoothing from the point n = P L is conducted 



Equation 22 

the point n = P L i 
according to the filter of Equation 23: 

E W + n = E W + n + 
n = 0,1 (PL/2). 

Equation 23 

20 In Equation 23, the value A is equal to 15/16 and H f jp L . 1 j = El 2 + 3 
* (EI^EIq). Thus, as indicated in Fig. 9, the extended sequence El n is 
substantially equal to L n on the left hand side, has a smoothed region 
beginning at the point P L and converges on the original shape of L^ 
toward the point 2P L - If L n was perfectly periodic, then El pL 1 = 

25 E, 'pL-r 

In the next step, the optimum match of R n with the vector El n is 

found. This match point is referred to as P (Block 305.) This is 

opt 

accomplished essentially as shown in Fig. 9 by comparing with El n 
to find the section of El n which most closely matches R n - This optimum 
30 blend point determination is performed using Equation 23 where W is 
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the minimum of PL and PR, and AMDF represents the average 
magnitude difference function. 

W-1 

AMDF<p) - 1 | El n + p -R n | 

5 n = 0 Equation 24 

This function is computed for values of p in the range of 0 to PL- 

1 . The vertical bars in the operation denote the absolute value. W is 

the window size for the AMDF computation. P Qpt is chosen to be the 

value at which AMDF(p) is minimum. This means that p = P Qpt 

10 corresponds to the point at which sequences El n + p (0<n<W) and 

R (0:sn<W) are very close to each other, 
n 

After determining the optimum blend point P opt * the waveforms 

are blended (block 306). The blending utilizes a first weighting ramp WL 

which is shown in Fig. 9 beginning at P Qpt in the El n trace. In a second 

1 5 ramp, WR is shown in Fig. 9 at the R trace which is lined up with P . . 

n opt 

Thus, in the beginning of the blending operation, the value of El n is 
emphasized. At the end of the blending operation, the value of R n is 
emphasized. 

Before blending, the length PL of L n is altered as needed to ensure 

20 that when the modified L and R are concatenated, the waveforms are 

n n 

as continuous as possible. Thus, the length P'L is set to P Qpt if P Qpt is 

greater than PL/2. Otherwise, the length P'L is equal to W + P Qpt and 

the sequence L^ is equal to El for 0^n:S(P'L-1). 
n n 
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The blending ramp beginning at P Qpt is set out in Equation 25: 



R n - El n + Popt + (R n " E, n + Popt )#<n + 1,/W °^ <W 



R = R Wrsn<PR 
n n 

Equation 25 

Thus, the sequences L n and R n are windowed and added to get 
the blended R n - The beginning of L n and the ending of R n are preserved 
to prevent any discontinuities with adjacent frames. 

This blending technique is believed to minimize blending noise in 
synthesized speech produced by any concatenated speech synthesis. 



10 IV. Pitch and Duration Modification (Fias. 10-18) 

As mentioned above with respect to Fig. 2, a text analysis 
program analyzes the text and determines the duration and pitch contour 
of each phone that needs to be synthesized and generates intonation 
control signals. A typical control for a phone will indicate that a given 

1 5 phoneme, such as AE, should have a duration of 200 milliseconds and 

a pitch should rise linearly from 220Hz to 300Hz. This requirement is 

graphically shown in Fig. 10. As shown in Fig. 10, T equals the desired 

duration (e.g. 200 milliseconds) of the phoneme. The frequency f^ is 

the desired beginning pitch in Hz. The frequency f is the desired 

e 

20 ending pitch in Hz. The labels P^ P 2 ..-,Pg indicate the number of 

samples of each frame to achieve the desired pitch frequencies f b , 

f 2 ...,fg. The relationship between the desired number of samples, P jf 

and the desired pitch frequency f. (f 1 = f b ), is defined by the relation: 

P. = F /f. r where F is the sampling frequency for the data, 
i s i s 

25 As can be seen in Fig. 10, the pitch period for a lower frequency period 
of the phoneme is longer than the pitch period for a higher frequency 
period of the phoneme. If the nominal frequency were Pg, then the 
algorithm would be required to lengthen the pitch period for frames 
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and ?2 and decrease the pitch periods for frames P^, Pg and Pg. Also, 

the given duration T of the phoneme will indicate how many pitch 

periods should be inserted or deleted from the encoded phoneme to 

achieve the desired duration period. Figs. 11 through 18 illustrate a 

5 preferred implementation of such algorithms. 

Fig. 1 1 illustrates an algorithm for increasing the pitch period, 

with reference to the graphs of Fig. 12. The algorithm begins by 

receiving a control to increase the pitch period to N + A, where N is the 

pitch period of the encoded frame. (Block 350). In the next step, the 

10 pitch period data is stored in a buffer x n (block 351). x r is shown in 

Fig. 12 at the top of the page. In the next step, a left vector L p is 

generated by applying a weighting function WL to the pitch period data 

x n with reference to A (block 352). This weighting function is 

illustrated in Equation 26 where M = N-A: 

15 L = x for 0<£n<A 

n n 



L = x * (N-n)/(M + 1) forA^n<N 
n n 

Equation 26 

As can be seen in Fig. 12, the weighting function WL is constant from 

the first sample to sample A, and decreases from A to N. 

20 Next, a weighting function WR is applied to x n (block 353) as can 

be seen in the Fig. 12. This weighting function is executed as shown 

in Equation 27: 

R = x a *(n + 1 )/(M + 1 ) forO<sn<N-A 
n n + A 

R = x A for N-A<;n<N. 

n n + A 

25 Equation 27 

As can be seen in Fig. 12, the weighting function WR increases 
from 0 to N-A and remains constant from N-A to N. The resulting 
waveforms l_ n and R n are shown conceptually in Fig. 1 2. As can be 
seen, L n maintains the beginning of the sequence x n , while R n maintains 

30 the ending of the data x n . 
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The pitch modified sequence y R is formed (block 354) by adding 

th two sequences as shown in Equation 28: 

y_= L + R, m » 
7 n n (n-A) 

Equation 28 

5 This is graphically shown in Fig. 12 by placing R n shifted by A below 

L . The combination of L and R shifted by A is shown to be y at the 
n n n 7 7 n 

bottom of Fig. 12. The pitch period for y n is N-f A. The beginning of 

y n is the same as the beginning of x^, and the ending of y n is 

substantially the same as the ending of x^. This maintains continuity 

10 with adjacent frames in the sequence, and accomplishes a smooth 

transition while extending the pitch period of the data. 

Equation 28 is executed with the assumption that L n is 0, for 

n^N, and R n is 0 for n<0. This is illustrated pictorially in Fig. 12. 

An efficient implementation of this scheme which requires at most 

15 one multiply per sample, is shown in Equation 29: 

y« = x « 0<n<A 
n n 

y n = x n + |x n-A' x n l#M + 1l/,N * A + 1) Asn<N 



N<;n<N d 

20 Equation 29 

This results in a new pitch period having a pitch period of N + A. 

There are also instances in which the pitch period must be 
decreased. The algorithm for decreasing the pitch period is shown in 
Fig. 13 with reference to the graphs of Fig. 14. Thus, the algorithm 

25 begins with a control signal indicating that the pitch period must be 
decreased to N-A. (Block 400). The first step is to store two 
consecutive pitch periods in the buffer x n (block 401 ). Thus, the buffer 
x as can be seen in Fig. 14 consists of two consecutive pitch periods, 
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with the period Nj being the length of the first pitch period, and N r being 

the length of the second pitch period. Next, two sequences L and R 

n n 

are conceptually created using weighting functions WL and WR (blocks 

402 and 403). The weighting function WL emphasizes the beginning of 

the first pitch period, and the weighting function WR emphasizes the 

ending of the second pitch period. These functions can be conceptually 

represented as shown in Equations 30 and 31, respectively: 

L = x n for 0<n<N. - W 

n n i 



L n = x n * (Nj-nJ/IW + D W:<n<N ( 



10 L n = 0 otherwise. 



and 



Equation 30 



R n = x n * (n-Nj + W-A+D/fW + D for N,-W + A<n<N,+ A 

R„ = x o for N. + A^n<N.-l-N 

n n I I r 

15 R^ = 0 otherwise. 

Equation 31 

In these equations, A is equal to the difference between IM ( and 

the desired pitch period N d - The value W is equal to 2*A, unless 2*A 

is greater than N rf , in which case W is equal to N d - 

20 These two sequences L and R are blended to form a pitch 

n n r 

modified sequence y n (block 404). The length of the pitch modified 
sequence y n will be equal to the sum of the desired length and the 
length of the right phoneme frame INK It is formed by adding the two 
sequences as shown in Equation 32: 

y n n (n + A) 

Equation 32 
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Thus, when a pitch period is decreased, two cons cutive pitch 
periods of data are affected, even though only the length of one pitch 
period is changed. This is done because pitch periods are divided at 
places where short-term energy is the lowest within a pitch period. 
5 Thus, this strategy affects only the low energy portion of the pitch 
periods. This minimizes the degradation in speech quality due to the 
pitch modification. It should be appreciated that the drawings in Fig. 1 4 
are simplified and do not represent actual pitch period data. 

An efficient implementation of this scheme, which requires at 
10 most one multiply per sample, is set out in Equations 33 and 34. 

The first pitch period of length N d is given by Equation 33: 

y n = x n 0<n<Nj-W 

= + [x n + A -x n ] # (n-N, + W + 1)/(W + 1) N f W^n<N d 

Equation 33 

1 5 The second pitch period of length N f is generated as shown in 

Equation 34: 

y n = X n-A + Ix n -x n A ]*{n.A.N, + W+1)/(W + 1) 

N|2Sn<Nj + A 

V n = x n N l + A sn<N l + N r 

2Q Equation 34 

As can be seen in Fig. 14, the sequence l_ n is essentially equal to 
the first pitch period until the point N ( -W. At that point, a decreasing 
ramp WL is applied to the signal to dampen the effect of the first pitch 
period. 

25 As also can be seen, the weighting function WR begins at the 

point N|-W + A and applies an increasing ramp to the sequence until 
the point N| + A. From that point, a constant value is applied. This has 
the effect of damping the effect of the right sequence and emphasizing 
the left during the beginning of the weighting functions, and generating 
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a ending segment which is substantially equal to the ending segment of 

x n emphasizing the right sequence and damping the left. When the two 

functions are blended, the resulting waveform y n is substantially equal 

to the beginning of x n at the beginning of the sequence, at the point 

5 Nj-W a modified sequence is generated until the point N ( . From N ( to the 

ending, sequence x n shifted by A results. 

A need also arises for insertion of pitch periods to increase the 

duration of a given sound. A pitch period is inserted according to the 

algorithm shown in Fig. 15 with reference to the drawings of Fig. 16. 

1 0 The algorithm begins by receiving a control signal to insert a pitch 

period between frames L and R (block 450). Next, both L and R 

n n n n 

are stored in the buffer (block 451 ), where L and R are two adjacent 

n n 

pitch periods of a voice diphone. (Without loss of generality, it is 
assumed for the description that the two sequences are of equal lengths 
15 N.) 

In order to insert a pitch period, x n of the same duration, without 

causing a discontinuity between L and x and between x and R , the 

n n n n 

pitch period x n should resemble R n around n = 0 (preserving L n to x n 

continuity), and should resemble L around n = N (preserving x to R 

n n n 

20 continuity). This is accomplished by defining x n as shown in Equation 
35: 

x n = R R + (L n - RJ * l(n + 1)/(N + 1)] 0<n<N-1 

Equation 35 

Conceptually, as shown in Fig. 15, the algorithm proceeds by 
25 generating a left vector WL(l_ n ), essentially applying to the increasing 
ramp WL to the signal L n - (Block 452). 

A right vector WR (R n ) is generated using the weighting vector 
WR (block 453) which is essentially a decreasing ramp as shown in Fig. 
1 6. Thus, the ending of l_ n is emphasized with the left vector, and the 
30 beginning of R is emphasized with the vector WR. 



WO 94/17518 



PCT/US94/00649 



- 29 - 

Next, WR (L ) and WR (R J are blended to create an inserted 
n n 

period x n (block 454). 

The computation requirement for inserting a pitch period is thus 

just a multiplication and two additions per speech sample. 

5 Finally, concatenation of L n# x n and R n produces a sequence with 

an inserted pitch period (block 455). 

Deletion of a pitch period is accomplished as shown in Fig. 17 

with reference to the graphs of Fig. 18. This algorithm, which is very 

similar to the algorithm for inserting a pitch period, begins with receiving 

1 0 a control signal indicating deletion of pitch period R n which follows L n 

(block 500). Next, the pitch periods L and R are stored in the buffer 

n n 

(block 501). This is pictorially illustrated in Fig. 18 at the top of the 

page. Again, without loss of generality, it is assumed that the two 

sequences have equal lengths N. 

15 The algorithm operates to modify the pitch period L which 

precedes R n (to be deleted) so that it resembles R n , as n approaches N. 

This is done as set forth in Equation 36: 

L '~ = L « + < R „- L J * Kn + 1)/(N + 1)] 0<sn<N-1 
n n n n 

Equation 36 

20 In Equation 36, the resulting sequence L' is shown at the bottom of 
Fig. 18. Conceptually, Equation 36 applies a weighting function WL to 
the sequence L n (block 502). This emphasizes the beginning of the 
sequence L n as shown. Next, a right vector WR (R n ) is generated by 
applying a weighting vector WR to the sequence R n that emphasizes the 

25 ending of R p (block 503). 

WL (L ) and WR (R ) are blended to create the resulting vector 
n n 

L' . (Block 504). Finally, the sequence L n " R n is replaced with the 
sequence L' in the pitch period string. (Block 505). 
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Accordingly, the present invention presents a software only text- 
to-speech system which is efficient, uses a very small amount of 
memory, and is portable to a wide variety of standard microcomputer 
platforms. It takes advantage of knowledge about speech data, and to 
5 create a speech compression, blending, and duration control routine 
which produces very high quality speech with very little computational 
resources. 

A source code listing of the software for executing the 
compression and decompression, the blending, and the duration and 

10 pitch control routines is provided in the Appendix as an example of a 
preferred embodiment of the present invention. 

The foregoing description of preferred embodiments of the present 
invention has been provided for the purposes of illustration and 
description. It is not intended to be exhaustive or to limit the invention 

15 to the precise forms disclosed. Obviously, many modifications and 
variations will be apparent to practitioners skilled in this art. The 
embodiments were chosen and described in order to best explain the 
principles of the invention and its practical application, thereby enabling 
others skilled in the art to understand the invention for various 

20 embodiments and with various modifications as are suited to the 
particular use contemplated. It is intended that the scope of the 
invention be defined by the following claims and their equivalents. 
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I. ENCODER MODULE 



#include <stdio.h> 
^include <math.h> 
^include <StdLib.h> 
^include <types.h> 
^include <fcntl.h> 
^include < string. h> 

#include <types.h> 
^include <files.h> 
^include <resources.h> 
#include <memory.h> 
#include "vqcoder.h" 

#def ine LAST_FRAME_FLAG 1 28 

#define PBUF_SIZE 440 

static float oc_state[21, nsf_state[NSF_ORDER + 1 ]; 

static short pstate[PORDER + 1 1, dstate[PORDER + 11; 

static short AnaPbuf[PBUF_S!ZE]; 

static short vsize, cbooksize, bs_size; 

#pragma segment vqlib 

/* Read Code Books */ 

float *EncodeBook[MAX_CBOOK_SIZE]; 

short # DecodeBookfMAX_CBOOK_SIZE]; 

get_cbook(short ratio) 

{ 

short *p; 

short frame_size, i; 
static short last_ratio = 0; 

Handle h; 
int skip; 

h = GetResource('CBOKM); 

HLock(h); 

p = (short •) *h; 

if (ratio = = last_ratio) 

return; 
last_ratio = ratio; 

if (ratio < 3) 
return; 



if (NOMINAL_PITCH < 165) 
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frame_size = 96; 

else 

frame_size = 160; 

get_compr_pars(ratio, frame_size, &vsize, &cbook_size, &bs_size); 
skip = 0; 

while (pfskip+1] != vsize) 
{ 

short t1, t2; 
t2 = plskip]; 
t1 = p[skip+1]; 

skip + = sizeof (float) * (2 * t2-1) * (t1 + 1) / sizeof (short) 
+ (2 * t2 * t1 + 2); 

} 

/*Skip Binary search tree */ 

skip + = sizeof (float) * (cbook_size-1 ) * (vsize + 1) / sizeof (short) 
+ (cbook_size * vsize + 2); 

/* Get pointers to Full search code books */ 

for (i = 0; i < cbook_size; i + + ) 

{ 

EncodeBookfi] = (float # ) &p[skipj; 

skip + = (vsize+1) * sizeof (float) / sizeof (short); 

} . 

for (i = 0; i < cbook_size; i + + ) 
{ 

DecodeBookli] = p + skip; 
skip + = vsize; 

} 

} 

char *getcbook(long *len, short ratio) 
{ 

get_cbook(ratio); 

*len = sizeof(short) * vsize * cbook_size; 

/* plus one is to make space at the end for the array of pointers V 
return (char*) DecodeBook[0]; 

} 

/* A Routine for Pitch filter parameter Estimation */ 

GetPitchFilterPars (x, len, pbuf, min_pitch, max_pitch, pitch, beta) 

float *beta; 

short *x, *pbuf; 

short min_pitch, max_pitch; 

short len; 

unsigned int * pitch; 

{ 

/* Estimate long-term predictor */ 
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int best_pitch, i, j; 

float syy, sxy, best_sxy = 0.0, best_syy = 1 .0; 
short *ptr; 

best_pitch = minjDitch; 

ptr = pbuf + PBUF_SIZE - min_pitch; 

syy = 1.0; 

for (i = 0; i < len; i+ + ) 
{ 

syy + = (*ptr) * (*ptr); 
ptr + + ; 

} 

for (j = min pitch; j < max pitch; j + + ) 
{ 

sxy = 0.0; 

ptr = pbuf + PBUF_SIZE - j; 
for (is0;i< len; i+ +) 

sxy + = x[i] * (»ptr+ + ); 

if (sxy > 0 && (sxy * sxy * best_syy > best_sxy * best_sxy * syy)) 
{ 

best_syy = syy; 
best_sxy = sxy; 
best_pitch = j; 

} 

syy . syy - pbuf[PBUF_SIZE - j + len - 1 ] • pbuf [PBUF_SIZE - j + len - 1 ] 
+ pbuf [PBUF_S!ZE - j - 1 ] • pbuf [PBUF SIZE - j - 1 ]; 

} 

•pitch = best_pitch; 

•bete = best sxy / best syy; 

} 

/* Quantization of LTP gain parameter •/ 
CodePitchFilterGain(beta, bcode) 
float beta; 

unsigned int * bcode; 
{ 

int i; 

for (i = 0; i < DLB_TAB_SIZE; i+ + ) 
{ 

if (beta < = dlb_tab[i]) 
break; 

} 

•bcode = i; 

} 

/* Pitch filter •/ 

PitchFilter(data, len, pbuf, pitch, ibeta) 
float *data; 
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short ibeta; 
short *pbuf; 
short len; 
unsigned int pitch; 

{ 

long pn; 
int i, j; 



j = PBUF_SIZE - pitch; 
for (i = 0; i < len; i + + ) 

{ 

pn = ((ibeta * pbuf(j++l) >> 4); 
datafij - = pn; 

} 

} 

/* Forward Noise Shaping filter */ 

FNSFilter(float # inp, float *state, short len, float *out) 

{ 

short i, j; 

for (j = 0; j < len; j+ +) 
{ 

float tmp = inp[j]; 

for (i = 1; i < m NSFJ3RDER; i+ +) 

tmp + = state[i] * nsfti]; 
outlj] = statelO) = tmp; 
for (i m NSF_ORDER; i > 0; i») 

state[i) = stated- 1]; 

} 

} 

/* Update Noise shaping Filter states */ 
UpdateNSFState(float *inp, float *state, short len) 
{ 

short i, j; 

float temp_state[NSF_ORDER + 1 1; 

for (i - 0; i < = NSF_ORDER; i + + ) 
temp_state[i] = 0; 

for (j = 0; j < len; j + +) 
{ 

float tmp = inplj]; 

for (i = 1; i < = NSF_ORDER; i+ +) . 

tmp + = temp_state[i] * nsfli]; 
temp_state[0] = tmp; 
for (i = NSF_ORDER; i > 0; H 

temp_state[i] = temp_state[i-1]; 
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} 

for (i = 0; i < = NSF ORDER; i+ + ) 
statefi] = stateli] - temp stated]; 

} 

/* Quantization of Segment Power */ 
CodeBlockGain(power, gcode) 
float power; 
unsigned int *gcode; 
{ 

int i; 

for (i = 0; i < DLG_TAB_SIZE; i+ + ) 
{ 

if (power < = dlg_tab[i]) 
break; 

} 

*gcode = i; 

} 

/* Full search Coder */ 

VQCoder(float *x, float *nsf state, short len, struct frame *bs) 
{ 

float max_x, tmp; 

int i, j, k, index, lshift_count; 

unsigned int gcode; 

float min_err = 0; 

max_x = x[0]; 
for (i = 1; i < len; i+ + ) 
if ( fabs{x[i]) > max_x) 
max_x = fabs(x[i]); 

CodeBlockGain(max_x, &gcode); 
max_x = qlg_tab[gcode]; 

lshift_count = 7 - gcode; /* To scale 14-bit Code book output to the 1 6-bit 

actual value */ 

bs-> gcode = gcode; 

for (i = 0; i < len; i + = vsize) 
{ 

/* Filter the data vector */ 
FNSFilter(&x[i], nsf_state, vsize, &x[i]); 

/* Scale data.*/ 
for (j = i; j < i + vsize; j+ +) 
x[j] = x[j] * 1024/max_x; 

index = 0; 

for (j = 0; j < cbook_size; j+ +) 

{ 
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tmp = EncodeBookljHvsize] * 1024.0; 
for (k = 0; k < vsize; k + +) 

tmp - = x[i + k] # EncodeBook[j][k]; 

if (tmp < minerr } | j = = 0) 

{ 

index = j; 
min_err = tmp; 

} 

} 

bs->vqcodeli/vsize] = index; 

/* Rescale data: Decoded data is 14-bits, convert to 16 bits */ 
if (lshift_count) 

{ 

for (k = 0; k < vsize; k + +) 

xli + k] = ((4 * DecodeBook[index][k]) >> lshift_count); 

} 

else 

{ 

for (k = 0; k < vsize; k + +) 

x(i + k] = 4 * DecodeBook(index][k]; 

} 

/* Update noise shaping filter state */ 
UpdateNSFState{&x[i], nsf_state, vsize); 

} 

} 

init_compress() 

{ 

int i; 

oc_statetO] = 0;; 
oc_state[1] = 0;; 
for (i = 0; i < = PORDER; i + +) 

pstate[il = dstateli) = 0; 
for (i = 0; i < PBUFSIZE; i++) 

AnaPbufli] = 0; 
for (i = 0; i < = NSF ORDER; i+ +) 

nsf_state[i] = 0; 

} 

Encoder(xn, frame_size, min_pitch, max_pitch, bs) 
short xn[l; 
struct frame *bs; 

short frame_size, min_pitch, maxjoitch; 
{ 

unsigned int pitch, bcode; 

float preemp_xn[PBUF_SIZE], beta; 

short xn_copy[PBUF_SIZE]; 
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short ibeta; 
float acc; 
int i, j; 



/* Offset Compensation */ 
for (i = 0; i < frame_size; i+ +) 

{ 

float inp = xnli); 

xn[i] = inp - oc_state[0) + ALPHA * oc_state[1]; 
oc_state[1] = xnfi]; 
oc state[0] = inp; 

} 

/* Linear Prediction Filtering */ 
for (i = 0; i < frame size; i+ + ) 
{ 

acc = pstate[01 = xn[i]; 

for (j = 1; j < = PORDER; j + +) 

acc-= pstatelj] * pfiltfj]; 
xn_copy[i) = preemp_xn[i] = acc; 
foMj = PORDER; j > 0; H 

pstatefjj = pstate[j-1]; 

}* 

GetPitchFilterPars <xn_copy, frame_size, AnaPbuf, min_pitch, 

max_pitch, &pitch, &beta); 
CodePitchFilterGain(beta, &bcode); 
ibeta = qlb_tab[bcodel; 

bs->bcode = bcode; 

bs-> pitch = pitch - min_pitch + 1; 

PitchFilter(preemp_xn, frame_size, AnaPbuf, pitch, ibeta); 

VQCoder(preemp_xn, nsf_state, frame_size, bs); 

/* Inverse Filtering */ 

j = PBUF_SIZE - pitch; 

for (i = 0; i < frame_size; i + + ) 

{ 

xn_copy[i] = preemp_xn[i); 

xn copy[i] + = ((ibeta * AnaPbuffj+ +]) > > 4); 

} 

/* Update Pitch Buffer */ 

j = 0; 

for (i = frame_size; i < PBUFSIZE; i+ + ) 

AnaPbuflj++] = AnaPbuf (i]; 
for (i = 0; i < f rame_size; i + + ) 
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AnaPbuf[j++] = xn_copy[i]; 

/* Inverse LP filtering */ 

for (i b 0; i < frame_size; i+ + ) 

{ 

acc = xn_copy[i]; 

for (j = 1; j < = PORDER; j++) 

acc = acc + dstatelj] # pfiltlj]; 
dstatelO] = acc; 
for (j = PORDER; j > 0; H 

dstatelj] = dstate[j-1]; 



for (j = 0; j < = PORDER; j+ +) 
pstatelj] = dstate[jl; 

} 

compress (short *input, short ilen, unsigned char *output, long *olen, long docomp) 

{ 

int i, j, vcount; 

unsigned char temp; 

short frame_size, minjpitch, max_pitch; 

if (docomp > 2) 
{ 

init_compress(); 

if (NOMINAL_PITCH < 165) 
{ 

min_pitch = 96; 
frame_size = 96; 
maxpitch = 350; 

} 

else 
{ 

min_pitch = 160; 
frame_size = 160; 
max_pitch =414; 

} 

bs_size = frame_size / vsize + 2; 

/* TEMPORARYfStoring State information */ 

pstatelU = * (input * 1); 

if (pstatelU > 0) 

pstate[1] = (pstatelU + 128)/ 256 + 128; 

else 

pstatelU = (pstatelU - 128) / 256 + 128; 



if (pstatelU < 0) 
pstatelU = 0; 
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if (pstate[1] > 255) 

pstate[1] = 255; 
♦output = pstatelU; 
J = 1; 

pstate[1] = pstatelU - 128; 
pstatelU = 256 * pstatelU; 
dstate[1] = pstatelU; 
/• End of Hack */ 

for (i = 0; i < ilen; i + = frame_size) 

{ 

Encoder(input + i, frame_si2e, min_pitch, max_pitch, output +j); 
j + = bs_size; 

} 

j - = bs_size; 

/• Number of vectors in last frame */ 

vcount a (ilen + frame_size - i + vsize - 1 ) / vsize; 

temp = outputljj; 

outputfj] = vcount + LAST_FRAME_FLAG; 
output [j + vcount + 21 = temp; 
*olen = j + vcount + 3; 

} 

else 
{ 

static long SampCount = 0; 
copy(input, output, 2*ilen); 
SampCount + = ilen; 
*olen = ilen; 

} 

} 

copy(a, b, len) 
short *a, # b; 
short len; 

{ 

int i; 

for (i = 0; i < len; i + + ) 
•b++ = Ca++); 

} 



u 
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II. DECODER MODULE 



^include < Types. h> 

^include < Memory. h> 

^include <Quickdraw.h> 

^include <ToolUtils.h> 

^include < errors. h> 

#include <files.h> 

#include "vtcint.h" 
#include <stdlib.h> 
#include <math.h> 
include <sysequ.h> 
#include < string. h> 

tfdefine MAX_CBOOK_SIZE 256 
#def ine LAST_FRAME_FLAG 1 28 

^define PORDER 1 

#define IPCONS 7 /• 7/8 */ 

#define LARGE_NUM 1 00000000 

#define VOICED 1 

tfdefine LEFT 0 
#define RIGHT 1 
^define UNVOICED 0 

#define PFILT_ORDER 8 

struct frame { 
unsigned gcode : 4; 
unsigned bcode : 4; 
unsigned pitch : 8; 
unsigned char vqcodel]; 

}; 

void expand(short **DecodeBook, short frame_size, short vsize, 
short min_pitch, struct frame *bs, short * output, short smpnum); 

get_compr_pars(short ratio, short frame_size, short * vsize, 
short *cbook_size, short *bs_size) 

{ 

switch (ratio) 

{ 

case 4: 

* vsize = 2; 
*cbook_size = 256; 
*bs_size = frame_size/2 + 2; 
break; . 
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case 7: 

*vsize = 4; 

*cbook_size = 256; 

*bs_size = frame_size/4 + 2; 

break; 
case 14: 

*vsize = 8; 

# cbook_size = 256; 

*bs_size = frame_size/8 + 2; 

break; 
case 24: 

*vsize = 16; 

*cbook_size = 256; 

*bs_size = frame_size/1 6 + 2; 

break; 
default: 

*vsize = 2; 

*cbook_size = 256; 

*bs_size = frame_size/2 + 2; 

break; 

} 

} 

short *Snlnit(short compratio) 
{ 

short * state, *ptr; 
int i; 

state = ptr m (short *)NewPtr((PFILT_ORDER + 1 + PFILT_ORDER/2 + 2) * 
sizeof(short)); 

if ( state = = nil ) 
{ 

return nil; 

} 

for (i = 0;i<PFILT_ORDER + 1;i++) 
*ptr+ + = 0; 

/* 

if (comp_ratio = = 24) 
{ 

*ptr+ + = 0.036953 * 32768 + 0.5; 
*ptr++ = -0.132232 * 32768-0.5; 
*ptr++ = 0.047798 • 32768 + 0.5; 
*ptr++ = 0.403220 * 32768 + 0.5; 
•ptr+ + = 0.290033 * 32768 + 0.5; 

} 

else 

{ 

*ptr+ + = 0.074539 * 32768 + 0.5; 

*ptr+ + = -0.174290 * 32768 - 0.5; 

*ptr++ = 0.013704 • 32768 + 0.5; 
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*ptr+ + = 0.426815 * 32768 + 0.5; 

*ptr+ + = 0.320707 * 32768 + 0.5; 

} 

*/ 

if (compratio = = 24) 
{ 

♦ptr+ + - 1211; 

*ptr+ + = -4333; 

*ptr+ + = 1566; 

*ptr+ + = 13213; 

*ptr+ + = 9504; 

} 

else 
{ 

*ptr+ + = 2442; 

*ptr+ + a -5711; 

*ptr+ + = 449; 

*ptr + + = 13986; 

# ptr+ + = 10509; 

} 

*ptr = 0; /* DC value */ 
return state; 

} 

SnDone(char * state) 

{ 

if ( state ! = nil ) 
{ 

DisposPtr(state); 

} 

} 

short **SnDelnit(p, ratio, frame_size) 
short *p,ratio, frame size; 

{ 

int i; 

short cbook_size = 256, vsize = 16, bs_size; 
short **DecodeBook; 

get_compr_pars(ratio, frame_size, &vsize, &cbook_size, &bs_size); 

DecodeBook = (short* *)NewPtr(cbook_size * sizeof (short*)); 
if (DecodeBook) { 

for (i sb 0; i < cbook_size; i + +) 

{ 

DecodeBookfi) = p; 
p + as vsize; 

} 

} 

return DecodeBook; 
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} 

SnDeDone(char *DecodeBook) 

{ 

if ( DecodeBook ! = nil ) 

{ 

DisposPtr(DecodeBook); 

} 

} 

void 

expand(short * *DecodeBook, short frame_size, short vsize, 

short minj)itch, struct frame *bs, short * output, short smpnum) 

{ 

short count; 

short *bptr, *sptr1, *sptr2; 
unsigned short pitch, bcode; 

/* 

short qlb_tab[] = { 

1,2,3, 4, 5, 6,7,8, 

9, 10, 11, 12, 13, 14, 15, 16 

}; 

•/ 

bcode = bs-> bcode; 

pitch = bs-> pitch + min_pitch - 1; 

/* Decode VQ vectors */ 

{ 

unsigned char *cptr; 
short k, vsize_by_2; 

short rshift_count = 7 - bs->gcode; /* We want the output to be 14-bit 
number */ 

sptrl = output + smpnum; 
cptr = bs->vqcode; 

vsize_by_2 = (vsize > > 1) + 1; /* +1 since we do a while H) instead of 
while (i») */ 

if (rshift_count) 
{ 

for {k = 0; k < frame_size; k + = vsize) 
{ 

bptr = DecodeBook[*cptr+ +J; 
count = vsize_by_2; 
while (-count) 
{ 

*sptr1 + + = (( # bptr++) >> rshift_count); 
•sptrl + + = ((*bptr + + ) > > rshift_count); 

} 

} 

} 
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else 

{ 

for (k = 0; k < frame_size; k + = vsize) 
{ 

bptr = DecodeBook[*cptr+ +]; 
count = vsize_by_2; 
while (-count) 

{ 

*sptr1 + + = *bptr+ +; 
*sptr1 + + = *bptr+ +; 

} 

} 

} 

} 

/* inverse Filtering •/ 
if (smpnum < pitch) 
{ 

sptrl = output + pitch; 

count = smpnum + frame_size + 1 - pitch; /* + 1 since we do a while H) 
instead of while (i-) */ 

sptr2 = sptrl - pitch; 

switch (bcode) 

{ 

case 0: 

while (-count) 

•sptrl + + + = ((*sptr2 + +) > > 4); 
break; 
case 1 : 

while (-count) 

*sptr1 + + + = ((*sptr2+ +) > > 3); 
break; 
case 2: 

while (-count) 

•sptrl + + + = ((3 • (»sptr2+ +)) > > 4); 
break; 
case 3: 

while (-count) 

# sptr1 + + + = ((*sptr2+ +) > > 2); 
break; 
case 4: 

while (-count) 

♦sptrl + + + = ((5 * (*sptr2+ +)) > > 4); 
break; 
case 5: 

while (-count) 

•sptrl + + + = ((3 • (*sptr2+ +)) > > 3); 
break; 
case 6: 

while (-count) 
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*sptr1 + + + = {(7 * ( # sptr2+ +)) > > 4); 
break; 
case 7: 

while (--count) 

*sptr1 + + + = ((*sptr2 + +) > > 1); 
breaks- 
case 8: 

while (--count) 
{ 

long tmp; 
tmp = *sptr2 + + ; 

*sptr1 + + + = (((tmp < < 3) + tmp) > > 4); 

} 

break; 
case 9: 

while (-count) 

*sptr1 + + + = ((5 * ( # sptr2+ +)) > > 3); 

break; 
case 10: 

while (-count) 

{ 

long tmp; 
tmp = *sptr2++; 

*sptr1 + + + = (((tmp < < 3) + 3 * tmp) > > 4); 

} 

break; 
case 1 1 : 

while (-count) 

*sptr1 + + + = ((3 * (*sptr2+ +)) > > 2); 

break; 
case 1 2: 

while (-count) 

{ 

long tmp; 
tmp = *sptr2++; 

*sptr1 + + + = (((tmp < < 4) - 3 * tmp) > > 4); 

} 

break; 
case 13: 

while (-count) 

*sptr1 + + += ((7 # (*sptr2++)) >> 3); 

break; 
case 14: 

while (-count) 

{ 

long tmp; 
tmp = *sptr2+ +; 

*sptr1 + + + = (((tmp < < 4) - tmp) > > 4); 

} 

break; 
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case 15: 

while (-count) 

*sptr1 + + += *sptr2++; 
break; 

} 

} else { 

sptrl = output + smpnum; 

sptr2 = sptrl - pitch; 

count as (f ramesize / 4) + 1 ; 

switch (bcode) 

{ 

case 0: 

while (-count) { 

♦sptrl + + + = ((*sptr2 + +) > > 4); 
*sptr1 + + += ((*sptr2++) >> 4); 
*sptr1 + + + = ((*sptr2+ + ) > > 4); 
*sptr1 + + += ((*sptr2+ + ) >> 4); 

} 

break; 
case 1 : 

while (-count) { 

•sptrl + + + = ({*sptr2+ + ) > > 3); 

*sptr1 + + + = ((*sptr2 + + ) > > 3); 

*sptr1 + + + = ((*sptr2++) >> 3); 

*sptr1 + + + = {(*sptr2 + + ) > > 3); 

} 

break; 
case 2: 

while (-count) { 

♦sptrl + + + a ((3 * (*sptr2+ +)) > > 4); 

*sptr1 + + + « ((3 * (*sptr2+ +)) > > 4); 

*sptr1 + + + o ((3 * (*sptr2+ +)) > > 4); 

*sptr1 + + + = {{3 * (*sptr2 + +)) > > 4); 

} 

break; 
case 3: 

while (-count) { 

*sptr1 + + + = ((*sptr2 + +) > > 2); 

*sptr1 + + + = (( # sptr2++) > > 2); 

*sptr1 + + += ((*sptr2++) >> 2); 

*sptr1 + + + = ((*sptr2++) >> 2); 

} 

break; 
case 4: 

while (-count) { 

*sptr1 + + + = ((5 * (*sptr2+ +)) > > 4); 

*sptr1 + + + = ((5 * (*sptr2+ + )) > > 4); 

*sptr1 + + + = ((5 • (»sptr2+ + )) > > 4); 

*sptr1 + + + = ((5 * (*sptr2+ + )) > > 4); 

} 
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break; 
case 5: 

while (--count) { 

*sptr1 + + + = {(3 * (*sptr2+ +)> > > 3) 
*sptr1 + + += ((3 * Csptr2+ + )) > > 3) 
*sptr1 + + + = ({3 * (*sptr2+ +)) > > 3) 
•sptrl + + + = ((3 * (*sptr2+ + )) > > 3) 

} 

break; 
case 6: 

while (-count) { 

*sptr1 + + + = {{7 ♦ (*sptr2+ +)) > > 4) 
*sptr1 + + + = {(7 * (*sptr2 + + )) > > 4) 

* sptrl + + + = {(7 • (*sptr2+ +)) > > 4), 

* sptrl + + + = {(7 # (*sptr2+ + )) > > 4), 

} 

break; 
case 7: 

while (-count) { 

*sptr1 + + += ((*sptr2++) >> 1); 

*sptr1 + + += ((*sptr2++) >> 1); 

# sptr1 + + + = ((*sptr2+ +) >> 1); 

*sptr1 + + + = <(*sptr2-f +) > > 1); 

} 

break; 
case 8: 

while (-count) { 
long tmp; 
tmp = *sptr2+ +; 

*sptr1 + + + = ((8 # tmp + tmp) > > 4); 
tmp = *sptr2 + + ; 

*sptr1 + + + = ((8 • tmp + tmp) > > 4); 
tmp = # sptr2+ +; 

•sptrl + + + = {(8 * tmp + tmp) > > 4); 
tmp = # sptr2++; 

*sptr1 + + + = ((8 * tmp + tmp) > > 4); 

} 

break; 
case 9: 

while (--count) { 

•sptrl + + += ((5 • (*sptr2+ + )) > > 3) 
•sptrl + + += {(5 • Csptr2+ + )) > > 3) 
•sptrl + + + = ((5 * Csptr2+ +)) > > 3) 
•sptrl + + + = ((5 • ( *sptr2 + + )) > > 3) 

} 

break; 
case 10: 

while (-count) { 
long tmp; 

tmp = *sptr2+ +; 
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*sptr1 + + + = (((tmp < < 3) + 3 * tmp) > > 4) 
tmp = *sptr2+ +; 

*sptr1 + + + = {((tmp < < 3) + 3 * tmp) > > 4) 
tmp = *sptr2+ + ; 

# sptr1 + + + = (((tmp < < 3) + 3 * tmp) > > 4) 
tmp = *sptr2+ +; 

# sptr1 + + + = (((tmp < < 3) + 3 * tmp) > > 4) 

} 

break; 
case 1 1 : 

while (-count) { 

*sptr1 + + + = ((3 • (*sptr2+ +)) > > 2); 

*sptr1 + + + = ((3 * (*sptr2+ +)) > > 2); 

*sptr1 + + + = ((3 * (*sptr2+ +)) > > 2); 

*sptr1 + + + = {(3 * (*sptr2+ +)) > > 2); 

} 

break; 
case 1 2: 

while (-count) { 
long tmp; 

tmp = *sptr2+ + ; 

*sptr1 + + + » (((tmp < < 4) - 3 * tmp) > > 4); 
tmp = *sptr2 + +; 

*sptr1 + + + = (((tmp < < 4) - 3 # tmp) > > 4); 
tmp = *sptr2+ +; 

*sptrl + + + = (((tmp < < 4) - 3 • tmp) > > 4); 
tmp = *sptr2+ + ; 

*sptr1 + + + = (((tmp < < 4) - 3 * tmp) > > 4); 

} 

break; 
case 13: 

while (-count) { 

*sptM + + + = ({7 * (*sptr2 + +)) > > 3); 

# sptr1 + + + « ((7 * (*sptr2 + +)) > > 3); 

*sptr1 + + + = ((7 * (*sptr2+ +)) >> 3); 

»sptr1 + + + = ((7 * (*sptr2++)> >> 3); 

} 

break; 
case 14: 

while (-count) { 
long tmp; 

tmp = *sptr2++; 

*sptr1 + + + = (((tmp < < 4) - tmp) > > 4); 
tmp = *sptr2+ + ; 

*sptr1 + + + = (((tmp < < 4) - tmp) > > 4); 
tmp = *sptr2 + +; 

*sptr1 + + + = (((tmp < < 4) - tmp) > > 4); 
tmp = *sptr2 + +; 

*sptr1 + + + = (((tmp < < 4) - tmp) > > 4); 
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break; 
case 15: 

while (-count) { 

*sptr1 + + + 

*sptr1 + + + 

*sptrl + + + 

*sptr1 + + + 

} 

break; 

} 

} 

} 

short SnDecompress(DecodeBook, ratio, frame_size, min_pitch, bstream, output) 

short "DecodeBook, ratio; 

unsigned char * bstream; 

short *output, frame size, min pitch; 

{ 

short count, SampCount; 
register short dstate; 
short vcount; 

short vsize, cbook_size, bs_size; 



= *sptr2++; 

= *sptr2 + + ; 

= *sptr2 + + ; 

= 'sptr2 + + ; 



get_compr_pars(ratio, frame_size, &vsize, &cbook_size, &bs_size); 

dstate = 'bstream + +; 
dstate = (dstate - 128) < < 6; 



SampCount = 0; 

while((* bstream & LAST_FRAME_FLAG) ===== 0) 
{ 

expand(DecodeBook, frame_size, vsize, min_pitch, 
(struct frame *)bstream, output, SampCount); 
bstream + = bs_size; 
SampCount + = framesize; 

} 

vcount = 'bstream - LAST_FRAME_FLAG; 
*bstream = '(bstream + 2 + vcount); 
expand(DecodeBook, frame_size, vsize, min_pitch, 
(struct frame *)bstream, output, SampCount); 
•bstream = vcount + LAST_FRAMEJ=LAG; 
SampCount + = vcount * vsize; 



count = (SampCount > > 1 ) + 1 ; 
while (-count) { 

*output+ + = dstate = ((IPCONS * dstate) > > 3) + 'output; 
*output++ = dstate = ((IPCONS * dstate) > > 3) + 'output; 

} 

output -= SampCount; 
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return SampCount; 

} 

^define FILTER state + PFILT_ORDER + 1 
^define DCJV/AL state + PFILT_ORDER + PFILTJ3RDER/2 + 2 
void SnSampExpandFilt(short *src, short off, short len, 
char *dest, short * state) 

{ 

short input, temp; 

long acc; 
register short dc = *(DC_VAL); 
register short *sptr1, *sptr2; 

src + = off; 
len + +; 
sptrl = state; 

sptr2 = state + PFILT_ORDER; 
while (-len) { 

input = *src + + - dc; 

dc + = input > > 5; 

temp = input + *sptr1 + +; /* (state[Ol + state|8]) * filter[0] •/ 
acc - temp * '(FILTER); 

temp = *--sptr2 + *sptr1 + +; /* (stated) + state[7]) * filter[1] */ 
acc +- temp * '(FILTER + 1); 

temp = *--sptr2 + *sptr1 + +; /* (state[2] + state[6]) * filter[2] */ 
acc + = temp * '(FILTER + 2); 

temp = *--sptr2 + *sptr1 + +; /* (state[3] + state[51) * filter[3] •/ 
acc + = temp * '(FILTER + 3); 

acc + = 'sptrl * '(FILTER + 4); /* state[4] ' filter[4] '/ 

if (acc > 0) 
{ 

temp = (acc + (257 << 20)) >> 21; 
if (temp > 255) 
temp = 255; 

} 

else 
{ 

temp = (acc + (255 << 20)) >> 21; 
if (temp < 0) 
temp = 0; 

} 

*dest+ + = temp; 
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sptrl - = 4; 
sptr2 - = 4; 

*sptr1 + + = *sptr2 + + ; 
*sptr1 + + = *sptr2+ +; 
*sptr1 + + = *sptr2 + + ; 
*sptr1 + + = *sptr2 + +; 
# sptr1 + + = *sptr2+ + ; 
*sptr1 + + = *sptr2 + + ; 
*sptr1 + + = *sptr2 + + ; 
*sptr1 = input; /* 
sptrl - = 7; 

} 

# (DC_VAL) = dc; 



/• statelO] = stated) */ 

/* stated) = state[2] •/ 

/* stateI2) - state[3) •/ 

/* state[3) = state[4] •/ 

/* state[4) = state[5] •/ 

/• state[5] - state[6] •/ 

/* state[6] = state[7] */ 
state[7) = input */ 
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III. BLENDING MODULE 



/* A module for blending two diphones */ 

typedef struct { 

short Iptr, pitch; 

short weight, weightjnc; 
} bstate; 

void SnBlend(pitchp Ip, pitchp rp, short cur_tot, short tot, 
short type, bstate *bs) 

{ 

#pragma unused (tot) 

short count; 
short *ptr1, *ptr2; 

if (type = = VOICED) 
{ 

if (cur_tot) 
return; 

{ 

short weight; 

long min_amdf; 

short bestjag = 0, lag; 

short window_size; 

short weightjnc; 

/* First replicate the left pitch period */ 

ptrl as lp->bufp; 

ptr2 = ptrl + lp->olen; 

count = lp->olen + 1; 

while (--count) 

*ptr2+ + = *ptr1 + +; 

/* Smooth the discontinuity */ 
{ 

register short en, e2; 

en = lp->bufp[2] + 

3 * 0p->bufplO] - lp->bufp[1]> - lp->bufp[lp->olen - 1]; 

e2 = lp->bufp[0] - lp->bufp[lp->olen - 11; 



if (en * en > e2 * e2) 
en = e2; 
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ptr2 = lp->bufp + lp->olen; 
count = (lp->olen >> 1) + 1; 
while (-count) 
{ 

# -ptr2 + = en; 

en = (((en < < 4) - en) > > 4); 

} 



min_amdf = LARGEJMUM; 

window_size = rp->olen; 
if (lp->olen < rp->olen) 

window_size = lp->olen; 

lag = rp->olen; 
while Hag) 

{ 

long amdf = 0; 

ptrl = rp->bufp; 

ptr2 = lp->bufp + lag; 

count = ((window_size + 3) > > 2) + 1 ; 

while (-count) 

{ 

short tmp; 

tmp = (*ptr1 - *ptr2); 
if (tmp > 0) 

amdf + = tmp; 

else 

amdf -= tmp; 
ptrl + = 4; 
ptr2 + = 4; 

} 

if (amdf < min_amdf) 
{ 

best Jag = lag; 
min_amdf = amdf; 

} 

} 

bs-> pitch = lp->olen; 

/* Update left buffer */ 

if (bestjag < (lp->olen >> 1)) 

{ 

/* Add bestjag samples to the length of left pulse */ 
lp->olen + = best lag; 

} 

else 
{ 

/* Delete a few samples from the left pulse */ 



ft 



>* 
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lp->olen = best Jag; 

} 

bs->Iptr = best Jag; 

weightjnc = 32767/ window_size; 

weight = 32767 - weightjnc; 

ptrl = rp->bufp; 

ptr2 = lp->bufp + bs->lptr; 

count = window_size + 1 ; 

while (-count) 

{ 

*ptr1 + + + = (((short) (*ptr2+ + - *ptr1) * weight) > > 15); 
weight - = weightjnc; 

} • 

} 

} 



register short delta; 

/* Just blend 15 samples */ 

ptr2 = lp->bufp + lp->olen - 15; 

ptrl = rp->bufp; 

/* 

f or (i = 1 ; i < 1 6; i + + ) 
{ 

*ptr1 m »ptr2 + (i # (*ptr1 - # ptr2)) > > 4; 
ptrl + +; 
ptr2 + + ; 

} 

•/ 

delta = *ptr1 - *ptr2; 

*ptr1 + + = *ptr2+ + + (delta > > 4); 

delta = *ptr1 - *ptr2; 

*ptr1 + + = *ptr2+ + + ((delta) >> 3); 

delta = "ptrl - *ptr2; 

•ptrl + + = *ptr2 + + + ((3 * delta) > > 4); 

delta = *ptr1 - *ptr2; 

*ptrl + + = *ptr2++ + (delta >> 2); 

delta = *ptr1 - # ptr2; 

*ptr1 + + = # ptr2+ + + ((5 * delta) > > 4); 
delta = *ptr1 - *ptr2; 

*ptr1 + + = # ptr2 + + + ((3 * delta) > > 8); 
delta = *ptr1 - *ptr2; 



♦ 
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*ptr1 + + = # ptr2 + + + ((7 * delta) > > 4); 

delta = *ptr1 - *ptr2; 

•ptr*l + + = # ptr2+ + + (delta >> 1); 

delta = *ptr1 - *ptr2; 

*ptr1 + + = # ptr2+ + + (((delta < < 3) + delta) >> 4); 
delta = *ptr1 - *ptr2; 

# ptr1 + + = *ptr2+ + + {(5 * delta) >> 3); 
delta = *ptr1 - *ptr2; 

# ptr1 + + = *ptr2+ + + (((delta < < 3) + 3 * delta) > > 4); 
delta = # ptr1 - *ptr2; 

*ptr1 + + = *ptr2+ + + ((3 * delta) > > 2); 
delta = *ptr1 - *ptr2; 

*ptr1 + + = *ptr2+ + + ({(delta < < 4) - 3 * delta) > > 4); 
delta = *ptr1 - *ptr2; 

*ptr1 + + = # ptr2 + + + ((7 * delta) > > 3); 
delta = # ptr1 - *ptr2; 

*ptr1 = *ptr2 + (((delta < < 4) - delta) > > 4); 



lp->olen - =15; 

} 

} 
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IV. INTONATION ADJUSTMENT MODULE 



/* A module for deleting a pitch period */ 
/* 

Pointer srd points to Left Pitch period 
Pointer src2 points to Right Pitch period 
Pointer dst points to Resulting Pitch period 
len = length of the pitch periods 

*/ 

skip_pulses(short *srd, short *src2, short *dst, short len) 
{ 

short i; 

register short weight, cweight; 
i = len+1; 

weight = cweight = 32767/i; 

while H) 

{ 

*dst+ + = # src1 + + + (((short) (*src2+ + - # src1) * cweight) >> 15); 
cweight + = weight; 

} 

} 

/* A module for Inserting a pitch period */ 
/• 

Locn bufferfcurbeg] points to Left Pitch period 

Locn buf fer[curbeg + curlen] points to Right Pitch period 

Pointer dst points to Resulting Pitch period 

curlen = length of the pitch periods 

*/ 

insert_pulse(short * buffer, short *dst, short curlen, short curbeg) 

{ 

short weight, cweight, count; 
short *src1, *src2; 

srd = buffer + curbeg; 

src2 = buffer + .curbeg + curlen; 

weight = 32767 / curlen; 

cweight = weight; 

count = curlen + 1; 

while (-count) 

{ 

*dst+ + = *src1 + + = *src2+ + + (((short) (*src1 - *src2) * cweight) > > 

15); 

cweight + = weight; 

} 

} 

/* This module is used to change pitch information in the concatenated speech */ 
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// This routine depends on the desired length (desien) being at least half 
// and no more than twice the actual length (ien). 

void SnChangePitch(short *buf, short *next, short Ien, short deslen,short Ivocshort 
rvocshort dosmooth) 

{ 

#pragma unused(rvoc, dosmooth) 
short delta; 
short count- 
short *bptr, *aptr; 
short weight, weightjnc; 
if (llvoc j | (desien = = Ien)) return; 

if (desien > Ien) 

{ . 

/* Increase Pitch period */ 
delta = desien - Ien; 
bptr = buf + Ien; 
aptr = buf + desien; 
count = delta + 1; 
while (-count) 

*-aptr = # --bptr; 

count = Ien - delta + 1 ; 

weight = weightjnc = 32767 / count; 

while (-count) 

{ 

register short tmp2; 

tmp2 = (*~aptr - # --bptr); 

•aptr = *bptr + ((tmp2 * weight) >> 15); 

weight + = weightjnc; 

} 

return; 

} 
{ 

/• Shorten Pitch Period */ 
short wsize; 

delta = Ien - desien; 
wsize = 2 * delta; 

if (wsize > desien) 
wsize = desien; 

weightjnc = 32767 / (wsize +1); 
weight = weightjnc; 
aptr = buf + desien; 
bptr = buf + len - wsize; 
count = wsize - delta + 1 ; 
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while (--count) 

{ 

*bptr+ + + = (((short) ( # aptr + + - # bptr) * weight ) > > 15); 
weight + = weightjnc; 

} 

aptr = buf + deslen; 
bptr = next; 
count = delta + 1; 
weight = 32767 - weight; 
while (-count) 
{ 

*bptr+ + + = (((short) ( # aptr+ + - # bptr) * weight ) > > 15); 
weight -= weight inc; 

} 

} 

} 
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CLAIMS 

What is claimed is: 



1 1 . An apparatus for synthesizing speech in response to a 

2 sequence of sound segment codes representing speech, comprising: 

3 memory storing a set of noise compensated quantization 

4 vectors; 

5 means, responsive to sound segment codes in the sequence, for 

6 identifying strings of noise compensated quantization vectors in the 

7 set for respective sound segment codes in the sequence; 

8 means, coupled to the means for identifying and the memory, 

9 for generating a speech data sequence in response to the strings of 

1 0 noise compensated quantization vectors; and 

11 an audio transducer, coupled to the means for generating, to 

12 generate sound in response to the speech data sequence. 

1 2. The apparatus of claim 1 , wherein the sound segment 

2 codes comprise data encoded using a first set of quantization vectors, 

3 and the set of noise compensated quantization vectors is different 

4 from the first set of quantization vectors. 

1 3. The apparatus of claim 1, wherein the noise compensated 

2 quantization vectors represent quantization of filtered sound segment 

3 data, and the means for generating a speech data sequence includes: 

4 means for applying an inverse filter to the identified strings of 

5 noise compensated quantization vectors in generation of the speech 

6 data sequence, wherein the inverse filter includes parameters chosen 

7 so that any multiplies are replaced by shift and/or add operations in 

8 application of the inverse filter. 
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1 4. The apparatus of claim 1 , wherein the noise compensated 

2 quantization vectors represent quantization of filtered sound segment 

3 data, and the means for generating a speech data sequence includes: 

4 means for applying an inverse filter to the identified strings of 

5 noise compensated quantization vectors in generation of the speech 

6 data sequence. 

1 5. The apparatus of claim 1, wherein the noise compensated 

2 quantization vectors represent quantization of results of linear 

3 prediction filtering of sound segment data, and the means for 

4 generating a speech data sequence includes: 

5 means for applying an inverse linear prediction filter to the 

6 identified strings of noise compensated quantization vectors in 

7 generation of the speech data sequence. 

1 6, The apparatus of claim 1 , wherein the noise compensated 

2 quantization vectors represent quantization of results of pitch filtering 

3 of sound segment data, and the means for generating a speech data 

4 sequence includes: 

5 means for applying an inverse pitch filter to the identified strings 

6 of noise compensated quantization vectors in generation of the 

7 speech data sequence. 
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1 7. The apparatus of claim 1 , wherein the quantization 

2 vectors represent quantization of results of pitch filtering and linear 

3 prediction filtering of sound segment data, and the means for 

4 generating a speech data sequence includes: 

5 means for applying an inverse pitch filter to the identified strings 

6 of quantization vectors in generation of the speech data sequence to 

7 produce a filtered data sequence; and 

8 means for applying an inverse linear prediction filter to the 

9 filtered data sequence in generation of the speech data sequence. 

1 8. The apparatus of claim 1 , wherein the means for 

2 generating a speech data sequence includes: 

3 means for concatenating the identified strings of quantization 

4 vectors and supplying the concatenated strings for the speech data 

5 sequence. 

1 9. The apparatus of claim 1 , wherein the identified strings of 

2 quantization vectors have beginnings and endings, and means for 

3 generating a speech data sequence includes: 

4 means for supplying the identified strings of quantization vectors 

5 for respective sound segment codes in sequence; and 

6 means for blending the ending of an identified string of 

7 quantization vectors of a particular sound segment code in the 

8 sequence with the beginning an identified string of quantization 

9 vectors of an adjacent sound segment code in the sequence to 

10 smooth discontinuities between the particular and adjacent sound 

1 1 segment codes in the speech data sequence. 

1 10. The apparatus of claim 1, wherein the means for 

2 generating a speech data sequence includes: 
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3 means, r sponsive to the sound segment codes for adjusting 

4 pitch and duration of the identified strings of quantization vectors in 

5 the speech data sequence. 

1 11. The apparatus of claim 1 , wherein the identified strings of 

2 quantization vectors have beginnings and endings, and means for 

3 generating a speech data sequence includes: 

4 means for supplying the identified strings of quantization vectors 

5 for respective sound segment codes in sequence; 

6 means for blending the ending of an identified string of 

7 quantization vectors of a particular sound segment code in the 

8 sequence with the beginning an identified string of quantization 

9 vectors of an adjacent sound segment code in the sequence to 

1 0 smooth discontinuities between the particular and adjacent sound 

1 1 segment codes in the speech data sequence; and 

1 2 means, responsive to the sound segment codes for adjusting 

1 3 pitch and duration of the identified strings of quantization vectors in 

14 the speech data sequence. 

1 1 2. The apparatus of claim 1 , further including an encoder 

2 including: 

3 a store for an encoding set of quantization vectors different from 

4 the set of noise compensated quantization vectors used in decoding; 

5 and 

6 means for generating the sound segment codes in response to 

7 the encoding set and sound segment data. 

1 13. The apparatus of claim 1 2, wherein the encoder further 

2 includes a linear prediction filter. 
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1 14. The apparatus of claim 12, wherein the encoder further 

2 includes a pitch filter. 

1 15. The apparatus of claim 12, wherein the encoder further 

2 includes a linear prediction filter and a pitch filter. 

1 1 6. An apparatus for synthesizing speech in response to a 

2 text, comprising: 

3 means for translating text to a sequence of sound segment 

4 codes; 

5 memory storing a set of quantization vectors; 

6 means, responsive to sound segment codes in the sequence, for 

7 identifying strings of quantization vectors in the set for respective 

8 sound segment codes in the sequence; 

9 means, coupled to the means for identifying and the memory, 

1 0 for generating a speech data sequence in response to the strings of 

1 1 quantization vectors; and 

12 an audio transducer, coupled to the means for generating, to 

13 generate sound in response to the speech data sequence. 

1 17. The apparatus of claim 1 6, wherein the sound segment 

2 codes comprise data encoded using a first set of quantization vectors, 

3 and the set of noise compensated quantization vectors is different 

4 from the first set of quantization vectors. 
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1 18. The apparatus of claim 16, wherein the noise 

2 compensated quantization vectors represent quantization of filtered 

3 sound segment data, and the means for generating a speech data 

4 sequence includes: 

5 means for applying an inverse filter to the identified strings of 

6 noise compensated quantization vectors in generation of the speech 

7 data sequence, wherein the inverse filter includes parameters chosen 

8 so that any multiplies are replaced by shift and/or add operations in 

9 application of the inverse filter. 



1 19. The apparatus of claim 16, wherein means for translating 

2 includes an table of encoded diphones, having entries including data 

3 identifying a string of quantization vectors in the set for respective 

4 diphones, and the sequence of sound segment codes comprises a 

5 sequence of indices to the table of encoded diphones representing the 

6 text; and 

7 the means for identifying strings of quantization vectors includes 

8 means responsive to the sound segment codes for accessing the 

9 entries in the table of encoded diphones. 

1 20. The apparatus of claim 1 6, wherein the quantization 

2 vectors represent quantization of filtered sound segment data, and 

3 the means for generating a speech data sequence includes: 

4 means for applying an inverse filter to the identified strings of 

5 quantization vectors in generation of the speech data sequence. 
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1 21. The apparatus of claim 16, wherein the quantization 

2 vectors represent quantization of results of linear prediction filtering 

3 of sound segment data, and the means for generating a speech data 

4 sequence includes: 

5 means for applying a inverse linear prediction filter to the 

6 identified strings of quantization vectors in generation of the speech 

7 data sequence. 

1 22. The apparatus of claim 1 6, wherein the quantization 

2 vectors represent quantization of results of pitch filtering of sound 

3 segment data, and the means for generating a speech data sequence 

4 includes: 

5 means for applying an inverse pitch filter to the identified strings 

6 of quantization vectors in generation of the speech data sequence. 

1 23. The apparatus of claim 1 6, wherein the quantization 

2 vectors represent quantization of results of pitch filtering and linear 

3 prediction filtering of sound segment data, and the means for 

4 generating a speech data sequence includes: 

5 means for applying an inverse pitch filter to the identified strings 

6 of quantization vectors in generation of the speech data sequence to 

7 produce a filtered data sequence; and 

8 means for applying a inverse linear prediction filter to the filtered 

9 data sequence in generation of the speech data sequence. 



WO 94/17518 



PCT/US94/00649 



- 67 - 



1 24. The apparatus of claim 1 6, wherein the means for 

2 generating a speech data sequence includes: 

3 means for concatenating the identified strings of quantization 

4 vectors and supplying the concatenated strings for the speech data 

5 sequence. 

1 25. The apparatus of claim 1 6 f wherein the identified strings 

2 of quantization vectors have beginnings and endings, and means for 

3 generating a speech data sequence includes: 

4 means for supplying the identified strings of quantization vectors 

5 for respective sound segment codes in sequence; and 

6 means for blending the ending of an identified string of 

7 quantization vectors of a particular sound segment code in the 

8 sequence with the beginning an identified string of quantization 

9 vectors of an adjacent sound segment code in the sequence to 

1 0 smooth discontinuities between the particular and adjacent sound 

1 1 segment codes in the speech data sequence. 

1 26. The apparatus of claim 1 6, wherein the means for 

2 generating a speech data sequence includes: 

3 means, responsive to the sound segment codes for adjusting 

4 pitch and duration of the identified strings of quantization vectors in 

5 the speech data sequence. 
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1 27. The apparatus of claim 16, wherein the identified strings 

2 of quantization vectors have beginnings and endings, and means for 

3 generating a speech data sequence includes: 

4 means for supplying the identified strings of quantization vectors 

5 for respective sound segment codes in sequence; 

6 means for blending the ending of an identified string of 

7 quantization vectors of a particular sound segment code in the 

8 sequence with the beginning an identified string of quantization 

9 vectors of an adjacent sound segment code in the sequence to 

10 smooth discontinuities between the particular and adjacent sound 

1 1 segment codes in the speech data sequence; and 

1 2 means, responsive to the sound segment codes for adjusting 

13 pitch and duration of the identified strings of quantization vectors in 

14 the speech data sequence. 

1 28. The apparatus of claim 16, further including an encoder 

2 including: 

3 a store for an encoding set of quantization vectors different from 

4 the set of noise compensated quantization vectors used in decoding; 

5 and 

6 means for generating the sound segment codes in response to 

7 the encoding set and sound segment data. 

1 29. The apparatus of claim 28, wherein the encoder further 

2 includes a linear prediction filter. 

1 30. The apparatus of claim 28, wherein the encoder further 

2 includes a pitch filter. 
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1 31 . The apparatus of claim 28, wherein the encoder further 

2 includes a linear prediction filter and a pitch filter. 

1 32. An apparatus for synthesizing speech in response to a 

2 text, comprising: 

3 a programmable processor to execute routines to produce a 

4 speech data sequence; 

5 an audio transducer, coupled to the processor, to generate 

6 sound in response to the speech data sequence; 

7 a table memory, coupled to the processor, storing a set of noise 

8 compensated quantization vectors, and a table of encoded diphones 

9 having entries including data identifying a string of noise 

10 compensated quantization vectors in the set for respective diphones; 

11 and 

12 an instruction memory, coupled to the processor, storing a 

1 3 translator routine for execution by the processor to translate text to a 

1 4 sequence of diphone indices, and a decoder routine for execution by 

1 5 the processor including 

16 means, responsive to diphone indices in the 

17 sequence, for accessing the table of encoded diphones to 

1 8 identify strings of quantization vectors in the set for 

1 9 diphones in the text; and 

20 means, coupled to the means for accessing and the 

21 memory, for retrieving the identified strings of quantization 

22 vectors; 

23 means, coupled with the means for retrieving, for 

24 producing diphone data strings in response to the identified 

25 strings of quantization vectors, wherein the diphone data 

26 strings have beginnings and endings; 

27 means, coupled to the means for retrieving, for 

28 blending the ending of a particular diphone data string in 
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29 the sequence with the beginning of an adjacent diphone 

30 data string in the sequence to smooth discontinuities 

31 between the particular and adjacent diphone data strings to 

32 produce a smoothed string of quantized speech data; and 

33 means, responsive to the text and the smoothed 

34 string of quantized speech data, for adjusting pitch and 

35 duration of the identified strings of quantization vectors for 

36 the diphones in the sequence to produce the sound data 

37 sequence for supply to the audio transducer. 



1 33. The apparatus of claim 32, wherein the sound segment 

2 codes comprise data encoded using a first set of quantization vectors, 

3 and the set of noise compensated quantization vectors is different 

4 from the first set of quantization vectors. 



1 34. The apparatus of claim 32, wherein the noise 

2 compensated quantization vectors represent quantization of filtered 

3 sound segment data, and the means for generating a speech data 

4 sequence includes: 

5 means for applying an inverse filter to the identified strings of 

6 noise compensated quantization vectors in generation of the speech 

7 data sequence, wherein the inverse filter includes parameters chosen 

8 so that any multiplies are replaced by shift and/or add operations in 

9 application of the inverse filter. 
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1 35. The apparatus of claim 32, wherein the quantization 

2 vectors represent quantization of filtered sound segment data, and 

3 the means for producing diphone data strings includes: 

4 means for applying an inverse filter to the identified strings of 

5 quantization vectors. 

1 36. The apparatus of claim 32, wherein the quantization 

2 vectors represent quantization of results of linear prediction filtering 

3 of sound segment data, and the means for producing diphone data 

4 strings includes: 

5 means for applying a inverse linear prediction filter to the 

6 identified strings of quantization vectors. 

1 37. The apparatus of claim 32, wherein the quantization 

2 vectors represent quantization of results of pitch filtering of sound 

3 segment data, and the means for producing diphone data strings 

4 includes: 

5 means for applying an inverse pitch filter to the identified strings 

6 of quantization vectors. 

1 38. The apparatus of claim 32, wherein the quantization 

2 vectors represent quantization of results of pitch filtering and linear 

3 prediction filtering of sound segment data, and the means for 

4 producing diphone data strings includes: 

5 means for applying an inverse pitch filter to the identified strings 

6 of quantization vectors to produce a filtered data sequence; and 

7 means for applying a inverse linear prediction filter to the filtered 

8 data sequence. 



WO 94/17518 PCT/US94/00649 

1/17 




r 



13 



DISPLAY 



AUDIO 
□UT 



^17 



ENCODED 
VOICE 
TABLES 



TEXT- 

TO- 
SPEECH 
CODE 



BUFFERS 



OTHER 
HOST 
MEMORY 



• TTS 
DICTIONARY 



• DIPHONE 
TABLE 



• NOISE SHAPED 
VECTOR QUANT- 
IZATION TABLE 

_ FOR ENCODING 

• NDISE COMPENSA- 
TED VECTOR 
TABLE FOR 
DECODING 



15 



FIG.-l 



SUBSTITUTE SHEET (RULE 26) 



a/17 



PCT/US94/00649 



RECEIVE INPUT TEXT 



TRANSLATE TD 
DIPHDNE STRINGS 



DECOMPRESS 
DIPHDNE STRINGS TD 
GENERATE VQ DATA FRAMES 



BLEND DIPHDNE 
VQ DATA FRAMES 



ADJUST 
DURATIDN DF DIPHDNE 
VQ DATA FRAMES 



ADJUST 
PITCH DF DIPHDNE 
VQ DATA FRAMES 



SUPPLY SPEECH DATA 
TD AUDID DUTPUT 



20 



TEXT - TD - SPEECH CDDE 



FIG. -2 



1_C 



22 



GENERATE 
INTDNATIDN 
CDNTRDL DATA 



J 



SUBSTITUTE SHEET (RULE 26) 



WO SM/17518 



PCT/US94/00649 



3/17 



Dlphone Record 




Left Dlphone 


Right Dlphone 


Left Pitch 


Right Pitch 


Period Couirt 


Period Count 


Pointer to Left 


Pointer to Right 


Pitch Period 


Pitch Period 


Pointer to Left 


Pointer to Right 


Demi Data 


Demi Data 



c 



34 



LP. 



LP 



LP NL-1 



Pitch Table 



£ 



36 



LFRAME i 



LFRAME 



LFRAME 



RFRAMEq 



RFRAME 



RFRAME MR _! 



VQ 

Compressed 
Speech 
Records 



VQ 

Compressed 
Speech 
Records 



RP< 



RP 



RP NR-1 



Pitch Table 



FIG.-3 



SUBSTITUTE SHEET (RULE 26) 



WO 94/17518 



4/17 



PCI7US94/00649 




Linear Predictive 
filtering 

Vn 

1 



Estimation of Prtch 
Filter parameters 
and Quan-tlza-tlon 

p opi * P 



I 



53 



Prtch Fll-ter 



I 



Block Gain Estimation 
G b; 



I 



54 



55 



56 



Full Search VQ Coder 



I 



Residual Coding Using _^ Store VQ String 



r 



57 



G ,0, P 



op-t 



Inverse Pitch Fll-ter 



58 



Pitch Buffer Update 
PBUF 



I 



59 



Inverse Linear 
Predictive Filtering 
<x_j Determination) 



60 



FIG. -4 



SUBSTITUTE SHEET (RULE 26) 



PCI7US94/00649 

5/17 



PBUF 1 y n 

100 101 




N-l 



max 



<3 



FDR ALL b 




APPLY NOISE 
FILTER AND 
SCALE Vjj 



180 



121 



123 



FIND POINTER 
TO BEST 
MATCH IN 
VECTOR 
QUANTIZATION 
TABLE 



ACCESS 
QUANTIZATION 

VECTOR 
USING POINTER 



USE FOR NEXT 
FRAME FILTER 
AND PBUF 
UPDATES 




185 



FIG. -6 



SUBSTITUTE SHEET (RULE 26) 



WO 94/17518 



6/17 



PCT7US94/00649 




200 



Decode 
Parameters 

G p op*t ' 
VQ string 



I 



Decode 
Residual 
Signal 



203 



202 



125 



ACCESS AND 




CONCATENATE 




QUANTIZATION 




VECTORS FOR 




VQ STRING 


« 



QV 0 
QVi 
QV 2 



QV 



255 



Inverse Pitch 
Filter 



/'~ N 204 



Sysnthesls Pitch 
Buffer Update 
SPBUF 



I 



Inverse Linear 
Predictive 
Filtering 



'205 



^^206 




FIG.-7 

SUBSTITUTE SHEET (RULE 26) 



7/17 



PCT/US94/00649 



RECEIVE LEFT AND 
RIGHT DIPHDNE 



300 



STORE LAST FRAME 
□F LEFT DIPHDNE 
IN BUFFER L n 



301 



STORE FIRST FRAME 
OF RIGHT DIPHDNE 
IN BUFFER R n 



I 



REPLICATE AND 
CONCATENATE L n 



302 



303 



SMDDTH 
DISCONTINUITY 

< El n > 



304 



I 



FIND OPTIMUM MATCH OF 
R n TD Et n 



305 



< P 



opt 



BLEND 
El n AND R n 
WITH P opi 



306 



FIG. -8 

SUBSTITUTE SHEET (RULE 26) 





FIG. -9 

SUBSTITUTE SHEET (RULE 26) 



WO 94/17518 



9/17 



PCT/US94/00649 



r » 





f 6 

i ! ! ! ! 






Ill 


► 




Pi Pa P3 P4 p s Pe 
. I . 





NOTES 1 

T = Desired duration of a phoneme 
f b = Desired Begin ing Pitch In Hz 

f = Desired Ending Pitch In Hz 
e 

PI, P2, . P6 are -the desired pitch period In 
No. of Samples corresponding to the frequencies 
fl,f2, . . .f6. 

Relationship between Pi and f I 1 

Pi = Fs/fl,where Fs is the Sampling frequency. 



FIG. -10 

SUBSTITUTE SHEET (RULE 26) 



10/17 



PCI7US94/00649 



INCREASE 


PITCH PERIOD 


TD 


N 


+ A 


i 


f , 



350 



STORE PITCH PERIOD 
DATA IN BUFFER 



351 



GENERATE LEFT 
VECTOR L n 
VL <x n ,A,N> 
(BEGINING MOST SIGNIFICANT) 



GENERATE RIGHT 

VECTOR R n 
VR <x n . N- A, N) 
(ENDING MOST SIGNIFICANT) 



352 



353 



BLEND L n AND R n -A 
L n + Rn-A 




FIG- 11 



SUBSTITUTE SHEET (RULE 26) 




FIG. -12 

SUBSTITUTE SHEET (RULE 26) 



WO 94/17518 



12/17 



PCI7US94/00649 



DECREASE 
TO 


PITCH PERIOD 
N - A 


1 


STORE 
PERIODS 


TWO PITCH 
IN BUFFER 




f 



GENERATE LEFT VECTOR 
L n = WL <x n , N r V) 

(BEGINING MOST SIGNIFICANT) 



400 



401 



402 



GENERATE RIGHT VECTOR 



403 



L n = WR (x n , N { + N r , V) 
(ENDING MOST SIGNIFICANT) 




404 



FIG. -13 



SUBSTITUTE SHEET (RULE 26) 



WO 94/17518 



13/17 



PCTAJS94/00649 



\ Weighting 
\Func-tlons 




FIG. -14 

SUBSTITUTE SHEET (RULE 26) 



WO 94/17518 



14/17 



PCT/US94/00649 



INSERT PITCH PERIOD 
BETWEEN L n AND R n 



450 



STORE 


L n AND R n 


IN 


BUFFER 



GENERATE LEFT 
VECTOR VL <L n > 
CENDING MOST SIGNIFICANT) 



GENERATE RIGHT 
VECTOR VR <Rn) 
(BEGINING MOST SIGNIFICANT) 



BLEND VR <L n ) AND VR <R n ) 
TO INSERTED PERIOD x n 



CONCATENATE 
Ln — * *n — Rn 



451 



452 



453 



454 



455 



FIG.-15 



SUBSTITUTE SHEET (RULE 26) 




FIG. -16 

SUBSTITUTE SHEET (RULE 26) 



WO 94/17518 



16/17 



PCT/US94/00649 



INSERT PITCH PERIOD 
R n WHICH FOLLOWS L n 



I 



GENERATE LEFT 
VECTOR WL <L n ) 
CBEGINING MOST SIGNIFICANT) 



STORE 


L n AND R n 


IN 


BUFFER 



500 



501 



502 



GENERATE RIGHT 
VECTOR WR <R n ) 
(ENDING MOST SIGNIFICANT) 



503 



BLEND WL <L n ) AND WR <R n ) 
TO CREATE RESULTING L' n 



504 



REPLACE L n — R n WITH 
L'n IN PITCH PERIOD STRING 



505 



FIG.-17 



SUBSTITUTE SHEET (RULE 26) 



F518 



PCT/US94/00649 



17/17 









N 


N 









Velglrtlng Function 




FIG. -18, 

SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



Intcmauo application No 

PCT/US 94/00649 



A. CIASSIFICA HON Ol- SUIUKCI' M ATIT.R 

IPC 5 G10L05/04 



According to Inlcmaoonal Patent Classification (iPQ or to both national classification and IPC 



II. MKI.DSSK ARCH KD 



Minimum documcntauon searched (classification system followed hy classification symbols) 

IPC 5 G10L 



Documentation searched other than minimum documentation to the extent that such documents arc included in the fields searched 



Klcctromc data base consulted during the inlcmauonal search (name of data base and. where pracucal, search terms used) 



C. DOCUMENTS CONSIDKRI-D TO III; RKU-VANT 



Category ' 



Citation of document, with indicauon, where appropriate, of the relevant passages 



Relevant to claim No. 



EP,A,0 515 709 (IBM) 2 December 1992 
see abstract 

see page 5, line 11 - page 7, line 23 
see figures 1,9,10 

US, A, 4 833 718 (R.P. SPRAGUE) 23 May 1989 
cited in the application 
see abstract 
see figure 4 
see claims 1-3 

US, A, 4 384 169 (F.S. MOZER ET AL.) 17 May 
1983 

cited in the application 
see abstract 
see claim 1 

-/— 



1.16,32 



1,16,32 



1,16,32 



13 



further documents arc listed in the continuation of box C. 



0 



Patent I ami I y members arc listed in annex. 



* Special categories of a ted documents : 

'A' document defining the general stale of the art which is not 

considered to be of particular relevance 
"15" earlier document but published on or after the mtcrnauonal 

filing date 

"I." document which may throw doubts on priority claimfs) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

'O' document referring to an oral disclosure, use, exhibition or 
other means 

*P* document published prior to the mtcrnauonal Itling date but 
later than the priority dale claimed 



* T" later document published after the mtcrnauonal filing date 
or prion ly date and not in conflict with the appitcauon but 
cited to understand the principle or theory underlying the 
invention 

"X" document ol particular relevance; the claimed invenuon 
cannot be considered novel or cannot be considered to 
involve an invenuvc step when the document is taken alone 

"Y* document of particular relevance; the claimed invenuon 
cannot he considered to involve an invenuvc step when the 
document is combined with one or more other such docu- 
ments, such combtnauon being obvious to a person skilled 
in the art. 

'&' document member of the same patent family 



Date of the actual completion of the mtcrnauonal search 



6 June 1994 



Date of mailing of the mtcrnauonal search report 



17.0B.9H 



Name and mailing address of the ISA 

Muropcan Patent Office, IMI. S81 8 PatcnUaan 2 
Nl. • 2280 I IV Rtiswiik 
I'd. ( i 31*70) 340-2040, Tx. 31 651 epo nl, 
I ax ( i 31-70)340-3016 



Authonzcd officer 



Daman, M 



Form PCTISA/210 (second iheel) (July 1993) 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 



IntcrnaUo application No 

PCT/US 94/00649 



(.•(Continuation) DOCUMKNTS CONSIDIiRIH) TO IIK RI-U-VANT 



Category " Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



WO, A, 85 04747 (FIRST BYTE) 24 October 1985 
cited in the application 
see abstract 
see claims 1,10 



1,16,32 



Form PCT/ISA'210 (cotiiinutUon of second sheet) (July 1992) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 

Infui.nauon on patent family members 



Intcmatic vpplica&on No 

PCT/US 94/00649 



Patent document 


Publication 


Patent family 


Publication 


cited in search report 


date 


member (s) 


date 


EP-A-0515709 


02-12-92 


JP-A- 5197398 


06-08-93 



US-A-4833718 


23-05-89 


US-A- 


4852168 


25-07-89 


US-A-4384169 


17-05-83 


US-A- 
US-A- 
US-A- 
US-A- 


4214125 
4458110 
4314105 
4384170 


22-07-80 
03-07-84 
02-02-82 
17-05-83 


W0-A-8504747 


24-10-85 


US-A- 
EP-A- 


4692941 
0181339 


08-09-87 
21-05-86 



Form PCT/ISAOI0 (patent family an nek) <Ju!y 1992) 



